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I INTRODUCTION 


1.1 INTRODUCTION AND HISTORICAL BACKGROUND 

Satisfactory control of multi-input, multi-output nonlinear system 
with process uncertainty is a very basic and challenging problem in 
systems science. The problem is made more complex by unavailability 
of states of the system necessary to implement the control and by noise 
associated with the observations. Several basic questions still to be 
answered in this context are: 

a) How large a control range is necessary to compensate for 
parameter uncertainty and unknown process nonlinearities? 

b) What is the minimum number of measurements necessary to 
generate the required control signal? 

c) What is an efficient way of controlling this class of 
systems and what is the relative effectiveness of various 
design techniques? 

The present thesis will deal with these problems to some extent and 
will propose various design procedures via optimal theory to generate 
the control. Since the system involves parameter uncertainty, consi- 
derable attention has been directed to implement the control via iden- 

[ 15 ] 

tification, usually known as adaptive control in the literature 
The fundamental concept was introduced possibly by Whitaker et . al^2,63j 
in 1958, followed quickly by Osburn et . al. . The basic idea is 
illustrated in Figure 1.1. The plant structure is assumed to be known 
but its parameters are unknown. The model generates the desired per- 


1 
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formance. The tracking error e is made small by adjusting controller 
parameters in an appropriate way. The adaptive analyser senses plant 
and model states and reference signal to provide proper variation of 
the controller parameters. The control structure design is based on 
the minimization of the integral square error between the plant and 
model outputs. This technique of adjusting controller parameters is 
referred to in the literature as the ’M.I.T.' rule. Similar techniques 
have been proposed by Krasovskii and may be found in the writings of 
Meerov^ 4 ®^. These adaptive schemes may possess severe instrumentation 
and stability problems. 

In another direction, linear regulator design has been shown to be 
an elegant design tool^ 4 The control function u, with | u^ | <_ 1, 
i = l,2,...n is selected to minimize the performance index 

00 

<Ku) = / x T Qx 

o 

subject to 

x = Ax + Bu. 

Q is positive definite and the unforced system, x = Ax, has been assumed 

[4 ] 

to be stable. Bass has also suggested a number of additional tech- 
niques for nonlinear regulator problems of the form: 
x = Ax + bf(o) 

where a is a linear feedback of all the state variables. This technique 

says nothing about stability of the given system since the control 

function f(o) decreases to zero as the system approaches equilibrium. 

[30] . Tipj 

Another interesting technique has been suggested by Lee and Geiss 
for systems of the form: 


x = f(x) + u 
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where the unforced system is stable, but not asymptotically. The 
resulting control u is designed to insure asymptotic stability, as well 
as to maintain a maximum control amplitude or design for minimum time 
to reach the origin x=o. An alternative approach to satisfactory solu- 
tion of this problem combines Lyapunov theory^wi^h the model-reference 
concept. A control signal is generated via a design scheme, using a 
dynamic model to guarantee that the plant follows the model. It is not 
necessary to identify the plant to implement the control strategy, 

although this technique may be used as parameter identification scheme 

r 53! 

as suggested by Rang . Rang assumes the process described by 
x = Ax + Bu 

where A and B are constant but unknown matrices. The model is taken 
to be asymptotically stable and of the form 

y = V + V* 

An error function defined as e = x-y satisfies the following 

e = A e + (A-A_)x + Bu - B r 
o 0 o 

A Liapunov function, 

V = e T pe +| | S^ T (D - A q )S 2 || + ||S 3 T (F - B Q )S 4 ||, 


is then chosen, where S„ , S„, S , S are constant column vectors and 

12 3 4 

D,F will be determined to approximate A and B by constraining the time 
derivative V of V to be negative definite and setting indefinite terms 
to zero. 

[21 223 

Grayson * synthesizes an algebraic (memory less) relay type con- 
troller by applying Liapunov's second method. Since Grayson's work, a 
number of generalizations and refinements of the technique have been 
made^ 5 * 4 ?-?a efforts mainly have been centered on the elimination of 
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higher order derivatives from the control signal. Motivation of this 
stems from the fact that it usually simplifies the hardware necessary 
to realize the controller and also reduces the noise level associated 
with differentiation involved in controller’s implementation. Monopoli^^ -4 ^ 

C37D 

and Lindorff have shown that, in some cases, the following possi- 
bilities exist: 

1) Some or all of the plant state variables may be replaced 
by the corresponding model state variables, 

2) the need for some higher order derivatives may be elimi- 
nated entirely, 

3) a reduction in the gain associated with the higher order 
derivatives will reduce the adverse effects of measure- 
ment noise . 

Lindorff has also successfully extended the Liapunov synthesis tech- 
nique to multivariable system when there are no input derivatives. 

A reduction in instrumentation noise level and the problem of noise 

[ 36 ] 

rejection has also been treated by Lindorff . This is especially 
directed to the systems which are not in phase variable form and where 
the parameter uncertainties exist. But it seems that the technique is 
restricted due to the requirement for special relationship. Non- * 

linearities in the system are not permitted. Confronted with the prob- 
lem of controlling a plant which is imperfectly identified, Taylor^ 5 ^ 
has obtained a realistic error bound for reduced order model-reference 
controller. Nikiforuk at. alp 1 *^ extended the model reference control 
synthesis technique to plant with unknown nonlinearities and unknown , 
parameters. The controller has been synthesized via a reduced order 
model. The resulting controller is highly nonlinear. The technique 
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is applicable to single-input, single-output minimum phase type systems. 
The limitation of the above techniques based on Liapunov's second method 
is not only the complexity of the controller structure, but also the 
lack of any insight as how to determine the control amplitude. 

Winsor and Roy^ combined optimal control and trajectory sensi- 
tivity to develop a design of desensitized model following control 
system. The control is generated by minimizing a quadratic performance 
index involving the error (between the plant and model output) and the 
control. Complete information regarding the plant, availability of all 
the state variables necessary to control the system, and model transfer 
relation being of same order as that of the plant, are assumed. 

A somewhat different approach has been suggested by Donalson and 
Leondes L . They have selected a variable controller (controller has 
variable parameters) which causes the form of the plant's transfer 
function to match that of the model. The control law is synthesized 
by minimizing a quadratic performance index involving the error and 
its derivatives by following the path of steepest descent for f(e). 

This method has produced a controller with memory , i . e . , the controller 

contains integrators. A discussion on overall stability of the system 

[14J 

has been suggested in [15]. Shackcloth and Butchart have also 
selected a variable controller by choosing a Liapunov function of the 
form: 

y = e T pe + Z T MZ 

where P and M are positive definite and symmetric and Z is the mis- 
alignment vector. The time derivative V or V» evaluated along the 
trajectory is constrained to be negative definite. Parks^ 48 ^ has also 
suggested a similar technique and has shown clearly that the adaptive 

Srfc 

Minimum phase type system implies the system with lef — half plane zeros. 
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technique based on M.I.T. rule might result to an unstable controller. 

He designes the controller so as to insure, under certain conditions, 
asymptotic stability. Many extensions and generalizations of the above 
technique have been made and may be found in the literature L •- 57 -i ’ 


1.2 PROBLEM STATEMENT AND ORGANIZATION OF WORK 


To counter various limitations of the techniques mentioned earlier, 
the present thesis deals initially with the problem of controlling a 
dynamic single input - single output system having parameter uncertainty 
using a minimax technique. Specifically, it treats the design of a con- 
troller using a nominal model to insure a satisfactory performance of 
the system in spite of ignorance of system parameters. The problem is 
posed with the additional constraint that the controller be linear and 
that it require only partial state feedback. 

The system is described by n th order differential equation 


(n) 

1 


+ a .x. 
n-1 1 


(n-1) 


+ a.x. = b + b 

o 1 m 

+bjU + b Q u + f (x^ 


ro -1 


» • • 


u 


(m-1) 

T t • i 

(n-m-1) 


.x 


1 


t) 


where x^ and u are the output and input to the system, superscript (n) 
denotes the number of derivatives, and f(x^,x , ...x^ n m 1 ,t) is a non- 
linear function. 

Next we define a lower order stable model 

(n-m) + _ y(n-m-l) + ( + a y = B r 

3 n-m-l-' o o 

where y and r are the model output and reference input respectively. 

A schematic diagram of this model reference system is shown in Figure 


2.2 
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Error e is defined as 
c l* x l x 


®_ = (x, ,x n ...x, n " m “ 1 ) T - (y,y,...y n-1 “~ 1 ) 


If the system is minimum phase type and f ( * ) is a bounded continuous 

nonlinear function, it is shown that the error e can be bounded with 

an arbitrarily small bound, despite imperfect knowledge of a. ,b, , i=0,l 

1 Jc 

. . . ,n,k=0,l, . . .m. Reduction of the error bound, however, requires in 
general greater control amplitudes. Furthermore this is achieved by a 
linear feedback obtained by minimizing with respect to control and 
maximizing with respect to a signal £ relating to the uncertainty, a 
quadratic performance criterion of the form 

J = i /°°[e T Qe + u T Ru - £ T L£ 3 dt . 

The resulting control is linear and the number of states required to 
generate the control is equal to system order less the number of zeros. 
This discussion is the subject of Chapter 2. 

One of the shortcomings of the minimax approach is that the control- 
ler requires the output of the plant and its derivatives up to (n-m-1). 
When some of the output derivatives are not available, a reduced order 
dynamic compensator is designed using a rainimax technique. The input 
to the dynamic system is the available states or output of the plant 
and its output is the required control signal. If the output observa- 
tion is noisy, differentiation of the output signal to generate the con- 
trol is no longer possible. 

In this case, estimation of states becomes complex due to lack of 
information regarding the system matrix. One way of approaching this 
problem is to obtain an optimal mean-square error estimate of the states 
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under specified parameter uncertainty . This class of adaptive 
estimation problem constitutes a class of nonlinear estimation prob- 
lems and the resulting estimator gains require the solution of a set 
of simultaneous partial differential equations. In most cases, a 
closed form solution does not exist and hence the estimator is diffi- 
cult to realize. Recently, a simplified closed form solution of this 

[30] 

type of adaptive filter has been suggested . This thesis reports 
an ad hoc estimation scheme to generate an estimate of the necessary 
states from the noisy observations using a deterministic model. The 
estimator is linear. Chapter 3 contains design of reduced order dynamic 
compensator and estimator for systems with parameter uncertainty. 

Another drawback of the present minimax procedure is the apparent 
difficulty in extending the concept to multivariable cases. The diffi- 
culty lies in the fact that a suitable canonical form for multivariable 
case is not currently available. The general situation becomes much 
more complex, due not only to multivariable nature of the problem, but 
also the presence parameter uncertainty, disturbance in the system and 
noisy measurement . One way of tackling the problem is to assume a form 
for the controller structure. Optimization techniques may then be 
employed to develop algorithms which yield optimum values of the para- 
meters of the controller. This thesis will assume a linear structure 


as illustrated in Figure 1.2, 

Athans, Levine^ ^oilnson^ancl Kosut^^ ^ have discussed a simplified 
version of the above problem. A dynamic compensator of specified order 
is used along with the output feedback to provide the control. Matrices 
G,H,P,N are chosen so as to minimize an integral quadratic criterion. 



Initial Condition i Disturbance 
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Figure 1.2 
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The result is a set of simultaneous nonlinear algebraic matrix equations 
which must be solved recursively for G,H,P,N. Basic limitations of 
their approaches are : 

1) A linearized system is assumed. 

2) Disturbances , cl , and measurement noise n are not con- 
sidered. 

3) Parameter uncertainty is not considered. 

4) Gain matrices depend on the initial state, of the 
plant. Generally this is handled in [33,28]by assuming 
that the initial state is random variable with a known 
covariance matrix . 

5) It is not generally known a prioiri whether a dynamic 
compensator can stabilize the system unless the dimension 
of and ^together equals that of x. 

Maclane^ 38 -^ has considered a stochastic version of the problem to 

handle the disturbance. The tracking problem can be treated within 

this formulation by assuming that the desired inputs may be generated 

by the initial condition response of a linear system. This, however, 

[231 

has not been done. Goldstein has designed a minimal order observer 
to yield an estimate of jc which, in turn, is used to obtain the control. 

No disturbance or measurement noise is allowed. Ferguson and Rekasius^ 18 ^* 
Pearson^ 88 ^, Pearson and Ding^ 8 ^, Brasch and Pearson^^ also use a 
dynamic compensator of suitable order to provide an optimal control. 

The gain matrices are independent of initial condition. 

The present thesis deals with the problem as schematically shown 
in Figure 1.2, Specifically, the problems of parameter uncertainty and 
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the presence of disturbance and measurement noise have been considered. 
The objective is to specify various gains G,H,P and N using optimiza- 
tion technique. The. initial step is to design a controller for a linear 
dynamic multi input - multi output system having parameter uncertainty 
using a minimax procedure. The basic study here is to examine various 
minimax criteria so that the system behaves acceptably well over a wide 
range of parameter variation using only output feedback. Thus the dee 
sign procedure involves the specification of G assuming H = 0. Mini- 
max controller design for this class of problems using complete state 

[ og 41 5fi 

feedback has been suggested by many authors ’ ' _ ’The present ap- 

58 ^ 6XJ • 

proach treats this problem by minimizing with respect to a feedback 
gain matrix and maximizing with respect to uncertainty, a quadratic 
performance index involving the system state, the control and a signal 
related to the uncertainty. The optimal feedback gain satisfies a set 
of nonlinear algebraic matrix equations. Several other minimax approa- 
ches are then considered to relax the conservativeness of the previous 
formulation. It has been demonstrated that the minimax design criteria, 
under wide parameter variation, yield better performance than a purely 
nominal design. This is the subject of Chapter 4. 

Next the thesis treats the problem of designing a generalized con- 
troller for systems excited by white noise disturbance. The measure- 
ments are assumed to be contaminated with white noise of known variance. 
It is well known that if the system is linear, is excited by white 
gaussian noise and the measurement noise is gaussian, the estimator 
and the controller can be designed independently. This is due to the 
well-known separation theorem. The estimator is the well-known Kalman 
filter whose dimension is equal to that' of the system. Sometimes the 
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dimensionality of Kalman filter restricts its use in practice because 
of added computational difficulty. With these points in perspective, 
the thesis deals with the design of a generalized controller (combined 
estimator and controller) of specified dimension. It is assumed that 
the system is perturbed by white noise and the output observations are 
contaminated with white noise signal of known variance. No parameter 
uncertainty is assumed. The dynamic linear controller operating on the 
available noise corrupted outputs of the system generates the required 
control input to the system. The design involves determination of 
optimal values of G,H,P and N. Chapter 5 deals with this problem and 
presents a simplified analysis of estimator and controller combined. 
Various special cases are also discussed. 

Chapter 6 summarizes the achievement of the present work, its 
shortcomings and possible extensions to more general problems. 
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II MINIMAX MODEL REFERENCE CONTROL IN SUBSTATE SPACE 
2.1 INTRODUCTION 

The design of a controller for a dynamic single input - single 
output system having parameter uncertainty will be undertaken in this 
chapter using a minimax technique. Specifically, it will be shown that 
a controller can be designed using a reduced order model to insure a 
satisfactory performance of the system in spite of ignorance of system 
parameters ? 8 ^The problem is approached by minimizing and maximizing 
with respect to the control and an "uncertainty signal" respectively, 
a quadratic performance index involving the tracking error, the con- 
trol, and the "uncertainty signal". The resulting controller is linear. 
The number of states required to generate the control is equal to the 
system order less the number of zeros. Bounded input - bounded output 
stability is guaranteed, provided the system transfer function in mini- 
mum phase type. The results also apply for systems with rather general 
nonlinearities that do not involve the control. It is also shown that 
the tracking error admits an upper bound and that the bound can be made 
arbitrarily small with adequate control levels. 


2.2 MOTIVATION OF THE PROBLEM 

Consider the single input - single output system described as in 
(2.1) - (2.3). 

X,(s) b s m + b 1 s m ‘ 1 + +b.s+b. N(s) 

m<n (2.1) 

U(s) s n + a n _i sn 1 + ....+ajS t a Q 



15 


where X^(s) and U(s) represent the Laplace transforms of output x^(t ) 
and input u(t) respectively. The equation (2.1) may be expressed in 

F3 i 

the state variable fornr J 



0 1 0 ... 0 i 


0 


001 .... 0 


0 


• • • 


0 

+ hu = 

• • • 

• • • 

x + 

h m 




n-m 


000 ... 1 


• 

4 

" a 0 -a l ” a 2 ••• * a n-l 


• 

h 

n 


u 


( 2 . 2 ) 


where h^ = 


k-1 

b n-k " J a n-k+i h i 


k<n-m 


k>n-m . 


i=n-m 


Since x^ +1 = x^, i=l,2, n-ra-1, it can be seen from (2.1) that 

N(s) 
sN(s) 


■ujfr ■ <SI - A) '^ * 


D(s7 


s^^Nis) 

N (s) 
n-m 


N n (s) 


(2.3) 


where N (s), k=0,l,...,m, are polynomials in s. Consider now the use 
of a linear feedback law 

u - - k Q [k^x^ + *2*2 + ’*** + ^n-m’Si-m^ = 2L (2.4) 

I 

where k T = [k, ,k_ k , 0, ....0] is a constant vector. 

— 12 n-m 
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The eigenvalues of the closed loop system are then solutions of 

0 = |sl - A + = [ sI-A | | I + k Q (sI-A) _1 hk T ! 

= | sI-A | (1 + (sI-A) _1 h) (2.5) 

where J A | is the determinate of matrix A. 

The last equality of (2.5) is obtained using the identity 

| I + C D T | = 1 + D T C ; (2.6) 

C_, £ being vectors of compatible dimensions . Furthermore , combining 

(2.3) and the definition of £ with (2.5) yields 

N(s)(k + koS + .... + k_ s 1 * - ® -1 ) 

|sI-A + kQldJl = D(s) [ 1 + k - 1 _ 3 

D(s) 


D(s) [1 + k 0 N l s I.yi L 3 = 0 
0 D(s) 


(2.7) 


(2.7) can also be obtained using Figure 2.1 which illustrates the 
system with feedback. The characteristic equation of the closed loop 
system is 

1 - Loop Gain = 1 + k^k^ (sI-A)”^£ =0, 
which agrees with (2.5) and (2.7). We know as k Q », that zeros of 
(2.7) approach the n-1 finite zeros of N(s)'k(s) and one zero at -®. 
Hence if N(s), k(s) are Hurwitz polynomials with b m , k n _ m > 0, the 
system is stable for k^ sufficiently large - regardless of the zeros 
of D(s) = | si— A | . Furthermore, response to any bounded input R(s) 
will be bounded. The problem is to choose the nonzero, elements of £ 
so that in addition to stability, the system exhibits behavior which 
is in some sense good. Furthermore we should like a design procedure 
which can yield this good behavior despite uncertainties in the system 
parameters and with minimal control effort. 



fc(s)=° X(s) 
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2.3 SYSTEM DESCRIPTION, DEVELOPMENT AND PROBLEM FORMULATION 


Define a stable 


i= v. + !o r = 


and a system 


model 

' 0 1 0 ... 

0 0 1 ... 

• • • 

• • • 

• • • 

0 0 0 . .. 

“ a 0 -Ct l _<X 2 *** 


0 


0 ' 

0 . 


0 

• 


• 

• 

0 


• 

• 

1 


0 

n-m-1 


*0 


( 2 . 8 ) 


x (n) + a +.... + a x. = b u m t b_ -.u 1 " -1 +...+b,u + b.u + 

1 n-1 1 0 1 m 1 0 

f(x 1 ,x 1> . . . ,x^ n m 1 ^t) (2.9) 

where jr_ and r are the model output and reference input respectively, 
and f ( • ) is a nonlinear function. A block diagram representation of 
the model reference system is shown in Figure 2.2. 


If 

(1) b >8>0 and b s m + b_ .s" 1- ^ + ...+b,s + b n is Hurwitz, and if 

m — ro m-i. l u 

(2) f(x^,x^,..., x^ m ^|t) is a bounded continuous nonlinear 

function, the objective now is to show that the error 

e = (x-j^jXj^,. . . - £ = x - £ (2.10) 

can be bounded with an arbitrarily small bound, despite imperfect 
knowledge of a^, b^, i = 0,l,...,n,k = 0,1,. ..m. This will be achieved 
by a linear feedback law which requires only partial state feedback and 
is in a sense optimal. Furthermore u will be similar in form to (2.4) 
with k(s) Hurwitz. 

Remark 2.1. 

a) n,m, are integers which may be unknown but the difference (n-m) 
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is assumed to be known. Also a lower bound of the highest order deri- 
vative coefficient of, u is available. 

b) Note that (2.9) is identical to the linear system (2.1) except 
for addition of the nonlinear term. 

c) When a regulator is being designed, it is admissable to let r=y_=0. 
We proceed by rewriting (2.9) to obtain 


m-1 


x l <n> + Vm-1 x l ( "‘ 1) + ••• + Vl < " > - Bu " * l b j u<3> + 

3~0 


(b -6) u m + £ a. x.^5) 

m .*• ]-m 1 — 

]=m 


n-1 


I 

3=0 


a j x l 


(j) 


+ f(x. 


’ x n-m t) 


Integrating each side m times, gives 


(2.11) 


(n-m) 


* “n-m-l x l 


(n-m-1) 


+ a 0 x 1 = 8u + 


5(t) 


( 2 . 12 ) 


where £(t) is the m fold integral of the right side of (2.11) together 
with initial condition terms. In state variable form, (2.12) may be 
written 


’ 0 1 0 . . . 0 


V 


0 

0 0 1 ... 0 


0 


0 

• • 

ft 

• 


• 

• 

x + 

• 

u + 

• 

• 


• 


• 

0 0 0 ... 1 


0 


0 

-a n -a. a ... -a 

, 0 1 n-n 

i-l 

. 6 . 


.5 ( t ). 


ft 

= Ax + £u + £(t) , 


(2.13) 
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* 

where jc corresponds to the first n-m elements of x as in (2.10) 
i • e . x - [Xj^x-l » . • • |X^ ] = tx^fXj , « . . ,x^_^j • ( 2 • 14 ) 

Consequently, subtracting (2.8) from (2.13) yields 
* - B 

e = (x - £) = A 0 e + £ (fl + 5), d = (u - _2.r),C = £. (2.15) 

ft 

?(t), of course, generates differences between the model andsx , 

If £=0=u, e(t) approaches zero asymptotically since A q has negative 
eigenvalues . Furthermore , even if u = 0, £ f 0 , e_ will be bounded for 
£(t) bounded. The problem is to realize a u which not only will retain 
stability but allow a bound on e_ to be made arbitrarily small. 

2.4 CONTROLLER DESIGN 

If the current and future values of £(t) were known for all time, 
the 'ideal 1 optimal control would be obtained by minimizing 
1 00 

J = =r f (e T Qe + Ru 2 )dt (2.16a) 

* o — — 

subject to (2.15), Unfortunately such a priori knowledge about £(t) 
is not available. Although it may be argued that £(t) can be generated 
from (2.15) once u is known, one still needs to know £(t) ahead of 
time to solve the optimization problem. If a bound on £(t) were known, 
one way of designing a minimax controller would be to maximize w.r.t. 
5(t) and then minimize w.r.t. u, the performance criterion (2.16a) 
subject to (2.15). But it is clear that an a priori bound on £(t) is 
difficult to ascertain. An indirect way of penalizing £ (t) can be 
achieved by modifying the criterion (2.16a) to include £. The form used 


will be 
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J = I f° (e^Qe + u 2 R - i 2 L)dt 

2 o 


(2.16b) 


with R, L > 0 and Q positive definite. u,£ will be chosen to minimize 
and maximize J respectively. Although £ is not arbitrary, the design 
assumes that £ acts in the worst possible fashion and thus maximizes J. 
The term -£ 2 L is introduced to limit £ . It is readily shown that 


the optimum u, £ are given by 


120 ] 


- It 'It 
u = - ± B/Pe, £ = - £ T Pe 

where P is the symmetric matrix satisfying 

PA + A'I'P + Q - PBB_ T P (I - I) * o 
0 0 R L 


(2.17) 


(2.18) 


1 T 

Furthermore , (i) A^ - — 88 P has negative eigenvalues 

(ii) P is the unique positive definite steady state 

solution of -P = PA + A T P + Q - PBB T P (~ - f), R<L. 

0 0 K L — 

* ft 

(iii) J[ e^; u,£ ] = min max J[ e ; u,£ ] = max min 

u £ £ u 

*V 


(2.17) will, in one sense, yield a conservative design since £ is assumed 
to act in the most perverse manner . In another sense , however , £ need 
not abide by the rules of the game and may be using a smaller L than 
assumed. This may require that R in turn be decreased. 

Remark 2.2 

It should be noted that a unique positive definite solution P of 

(2.18) exists^ if R<L. In case, R<L the implication is that we are 
trying to balance the effect of the "uncertainty signal" with a larger 
amplitude of control signal than what is needed with R = L. Consequently 
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for much of the rest of the chapter, we shall restrict ourselves to the 
case R = L. In case R<L, it is still possible to show that proposi- 
tion 2.1 (mentioned below) is true. In case R>L, nonuniqueness of 
Solution P of (2.18) poses some problem in the subsequent analysis . 
of the error bound and is a subject of further investigation. 

For convenience, define )< as the last row or column of P. Then 

P J*. s £ 2 » E^-m-IE 00, * * ^ = SE^-m " $S* , ■ 


From (2.17), u is given by 




(219a) 


Accordingly 

3(s) = -|k T [l,s...,s n " m “ 1 ] T E(s) = - |k(s)E(s) (2 . 19b) 

where 

, \ T r . 2 n-m-1-.T 

k(s) = [l,s,s ,...,s J . 

Now it will be helpful, at this stage, to establish an important 
property of this control signal u that is outlined in the following 
proposition: 

Proposition 2.1 

With Aq stable and as defined in (2.13), k(s) is a stable poly- 
nomial for R=L. 

Proof ; 

Consider the system defined by (2.15) and represented in Figure 
C2.3), The open loop transfer function, z(s), between £(s) and u(s), 
is given by 
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z(s) = [ |p (sI-A ) ^B] = — 6 T P/D-| (s) [1,s. . . .s* 1- ®' 1 ] = — — ^ 

R R ~ R D 1 (s) 

( 2 . 20 ) 

where D^(s) = [ si— A Q | is a stable polynomial and k(s) is as defined 

in (2.19b). Now using (2.18) we obtain 

2Re[z(s)] =z(s) + z(s) = ~ [B T (sI-A^) 1 P8 + $ T P (sI-A.)"^] 

= £ T (sI-aJ) _ 1 [P(s+s) + PB. T (R -1 -L -1 ) 0P + Q](sI-A 0 ) _1 B_ 

= 2 {Res} CB T (sI-A T )" 1 P(sI-A n ) _1 B] + B T (sI-A T )* 1 Q(sI-A n )‘ 1 B 

— 0 u — — 0 u — 

for R=L. 

A 

Here s denotes the complex conjugate of s. Since P, Q are positive 
definite, Re[z(s)] is nonnegative for Re(s)>0. Therefore the trans- 
fer function z(s) is positive real^ 1 ^ which implies from (2.20) that 
k(s) is Hurwitz. 

2.5 DERIVATION OF A BOUND ON THE ERROR 

In order to determine a bound on £ when u satisfies (2.17) but £(t) 
is arbitrary, let 

V(e) = -ie T Pe (2.21) 

2~ - 

where P is the positive definite matrix satisfying (2.18). e^ must 
now satisfy the differential equation (2.15) with u as in (2.19a); 
that is 

e = [A - --B T P] e + B £(t). (2.22) 

— Or— - - 

The time derivative of (2.21) is 
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tf(e) = [A n - - 88 P) P + P (A - i B 8 P)2] e + C(t)8 Pe 
2 OR — OR — — — 


1 T„ 


*Qe - 7 6 2 (k T e) 2 (i + h + K (t)8(k T e) (2.23) 

2— — 2 L R 


where (2.18), (2.19a). and (2.22) have been used to refine the result. 
Clearly if |5 (t)| is bounded, (2.23) will be negative for | | e_| J 
sufficiently large and admit an upper bound | | e | | . It is necessary 
to examine £(t), therefore, by considering the behavior of the full 
system described by (2.9), (2.10), (2.21). It will then be possible 
to complete examination of (2.23). 

2.6 STABILITY OF THE OVERALL SYSTEM 

Turning now to the total system, (2.9) may be written in state 
form basically as in (2.2, (2.14), 

A 

x = Ax + hu + £ (x, t) (2.25) 

with f_ (x_,t) = [0,0 ,. . . ,0,f(x,t)D T . 

Now define 


* 

x = [-], A = 
— v 


- A v* 


"V 


V -i 


(2.26) 


Z = [v], v = (v 1# v 2 , v m ) 

* 

and combine (2.25), (2.8) and (2.10), to eliminate x^ . The result is 

r + f (e + y, t) 



A -A ' 


fe * 

= AZ + hu + 

* 0 

z - 

•HD 


A 


0 


L v ft J 




= AZ + hu + v 


(2.27) 


where 
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n-m n-m T 

v = [ 0 , 0 , ..., 0 , l a. y. - fi r, 0 , 0 , . ,f + e, t ) - l a. .y.j . 

i =0 i=l 1 “- L x 

(n-m)* th entry (2.28) 

v_ is bounded if the model input r, is bounded as assumed. If the 

"optimal" u given by (2.19a) is used in (2.27), the closed loop 

system satisfies 


[A-|-hk T ]Z + h 


8 . 

— *■ r + \> 
8 


fi ~T 

= [A - - h k ] Z + v ’ 

with 

k T = [k , k_,....k , 0 , 0 ,.., 0 ] 

— 12 n-m 

B 0 A 

v' = v + h — r = [0,0,... 0, v' ...v'3 
— — — fi n-m n 


(2.29) 


(2.30) 


The characteristic equation of the closed loop system is obtained 
as in (2.5) - (2.7), i.e. 


0 = |sI-A + 7 h k T | = | sI-A] |l + (sI-A) -1 - h k T | 

R R 


= J sI-A j (1 + |-k T (sI-A) 1 h) 
R 

= D(s ) [I + i 2 <i 2 . (V + vs ♦ .... + k 

R D(s) 1 2 n-m 


= D(s) + ~ N(s) k(s) 


(2.31) 


Now as R-K), (n-1) roots of (2.31) approach zeros of N(s)k(s), and 
the last root goes to infinity along the negative real axis. Since 
N(s), k(s) are "stable polynomials", then (2.30) is stable for R 
adequately small. 



27 


Consequently as the penalty R on control is reduced, permitting 

larger control amplitudes, the system (2.1) - (2.2) or (2.7) is stable 

for the feedback law (2.19) provided N(s) is a stable polynomial with 

b >0. 
m 

Now since (2.29) yields a bounded Z with a bounded input , all 
the elements of Z_ are bounded . In fact , bounds on v_* do not depend 
on Z but are determined mainly by £. From (2.29) - (2.31) it can be 
seen that 

i n • 

£(e.) =E. (s) = \ fT(s) v!(s) i=l,2,. . . ,(n-m-l) 

1 1 D(s) + _ N(s)k(s) j=n-m ^ 

R 

£(v.) V.(s) = l Cg^(s) + — J^(s)]v’.(s), 

1 1 D(s) + N(s)k(s) j=n-m ^ R ] 

R 

^2 j • • • )in* 

# ^ £ 

where f^s), g 1 (s) and £.(s) are polynomials in s, independent of R, 

3 3 3 

and of order <_ (n-1). £(') is the laplace transform of CO. 

As R-*-0, J E^(s ) J-*- 0, i=l,2 , . . . ,(n-ra-l) 

and 

] V . ( s ) | l SL X .(.s) v’(s) 

1 j=n-m - 1 J i=l,2,...,m. Thus an ultimate 

N(s)k(s) 

bound on exists and is independent of R as R->0. Furthermore the 
error bound, i.e. , the bound on | |ej | goes to zero as R-*-0. We 
explore this further. 

2.7 FURTHER RESULTS ON THE BOUND OF THE ERROR 
Let us now examine £(t). From (2.22) 

n-m 2 T 

S(t) = 8€(t) = e + J a. .e. + — k e 
n-m l=i 1 R 
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n-m 


= e 


:• p 02 

e_ + I a e . + — k. )e . 
n-m i-l i r i l 

From (2.29), on the other hand 


g T _ 

i = 6 m = v. - — b k e+ T a y. - 8 r + b Or. 

n-m n-m 1 r m l i_ 2. i 0 m — 

i-o g 


n-m 


8 


Thus 


n-m 


5(t) = v + J [a. .e . + o. .y.] +8 ( — -1) r + — (B-b )k e 

1 j_sl 1-1 i 1-1 1 0 g R m 


= £. + i (8-b )k?i 

1 r m — — 


(2.32) 


where 


n-m 


C = v + l [a. e. + a. y.] + 8 ( ■— -l)r 
1 1 i-l i i-l i 0 B 


has an upper bound which is independent of R and exists as R-*0. 
Returning to (2.23) with (2.32) replacing £(t). 


V(e) = 




< - i e T Qe - i (k T e) 2 8 2 (J + f ) + 5,k T e 

< - ~ e T Qe - - (k T e ) 2 B 2 (“ + f ) + |?,| | k T e| 

— 2 — — 2 r L ' 1 1 max ' 1 


(2.33) 


-hw p r? - -«s? 

2 <H )6 


The last inequality is obtained by maximizing the last two terms in 

T 

the equality with respect to |k.ej. 

Providing e_ is sufficiently large, the first component of w(e) 

1 T 

will dominate. Next £ is maximized subject to w=0, in order to 

T 

establish an upper bound on e e. For this purpose , it i s required 
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to maximize the Hamiltonian 


H = i e T e + nC - i e T Qe t c], c = i 1^1 


(2.34) 


where n is a constant multiplier. Equating to zero yields 

o6 


Qe = - e = X-e 

~ n — Q - 


(2.35) 


Thus (2.35) shows that e^ and ^ are an eignevector and eigenvalue 
of Q respectively. Since 


T T 

eQe = 2c = X^e 


is required t 


* 2C/ < x q) " in 


where (X.) . is the minimum eignevalue of Q. 
Q mxn 


* T • s l 1 may 

Thus V < 0 for all e e > — - ■ 


<i ♦ i> 6 * <Vmin 


i.e,, V < 0 for all | |e_| | >_ R = 




max 


0 


(X.) . / X 1 . 

Q min < r + 


(2.36) 

This, of course, is conservative. Since we have permitted e to both 
maximize certain terms in (2.33) and to maximize | [ej | subject to 
W.(e) >_ 0. 

The bound on | |ej | can now be found in standard fashion, 

(1) Determine V = maximum V(e) subject to ||e|| <_ R 

(2) Determine = max ||e|| subject to V(e) <_ V q 


i 
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Thus , to find V Q , we proceed as follows : 
Minimize the Hamiltonian H given by 


H = ejpe_ + v^CR^ - e^e] 


2 T 


or Pe = v^e_ 


indicating that e_ and are eigenvector and eigenvalue of I 


Thus ^_ T Pe^ = v e^e. = v^R a 2 


V o 5 = < l p>max R a 2 


where (X )__„ is the maximum eigenvalue of P. 

p ID 3.X 

o T 

Now to find R^, we maximize R^ = £ e. subject to 


£ Tp e = (A p ) max R a 2 - 
H = £ T e + v 2 [(X p ) max R a 2 - e T Pe 3 


and 1— = 0 = e-*v_Pe 

OQ — ^ — 


or Pe = — . e . 

v 2 - 


Hence e and — are eigenvector and eigenvalue at P. 
v 2 

T 2 T 

Furthermore e Pe = (X_)_ R = X e e 
— — p max a p 


i.e. 


e T e = (X ) v R_ 2 / x 
— — p max a 


Thus the required' R^ is given by 


2 ^p^max R a 

* '"V^T 


(2.37) 


respectively. 


(2.38) 


(2.39) 


(2.40) 
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and 


*b ■ R , 




/ (X ) . 

P nan 


“l'max 

6 


V — — — 

f (X 0 ) . ,11. 

Q' mxn(- + -) 


(2.41) 


R^ represents an ultimate bound on | |ej | since e^ asymptotically 

approaches the region defined by v(e) <_ v q but v(e) may be indefinite 

therein. Thus | |ej | is bounded by as shown in Figure (2.4). This 

bound can be made arbitrarily small by relaxing the penalty R on the 

control u since I 5, I admits an upper bound as can be seen from 
1 1 1 max 

the previous section and (2.32). 


2.8 EXAMPLES 

To illustrate the preceding analysis, consider the following open 
loop unstable system described by 

*x* + x - x + x - sin x + x 2 = u* + 2u + u (2.42) 

with a first order model described by 

y + 2y = r (2.43) 

where r is a step input. It is to be noted that the minimum order of 
the model is specified by the difference between the number of poles 
and zeros of the plant. This, in this example, is one. With a little 
manipulation and integrating (2.42) twice, the error equation can be 
expressed as 

e + 2e = E + u (2.44) 

a = u - r 
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* 

It is desired to find an optimal control u by maximizing w.r.t. £ 
and minimizing w.r.t u the performance criterion 

J = ~ f (4e 2 + RS 2 .- L£ 2 ) dt, R=L (2.45) 

subject to (2.44). 

The resulting control is given by 

S = -R _1 e (2.46) 

and is applied to the original system (2.41). 

The outputs of the plant and model are shown in Figure (2.5) for 
different values of R. It can be seen that the error decreases mono- 
tonically with the decrease of penalty R. Also plotted are the re- 
maining states x 2 = x, x 3 = x* for different values of R . These 
are bounded as can be seen in Figure (2.7) and (2.8). The control 
signal characteristics are shown in Figure ( 2 . 9 ) . 

2.9 CONCLUSIONS 

A linear time invariant controller has been designed for a single 
input - single output system with parameter uncertainty. The number 
of states required to generate the control signal is equal to the 
system order less the number of zeros. The feedback gains are ob- 
tained by minimizing a quadratic performance index involving the 
tracking error, the control signal and the "uncertainty signal". 

This, however, yields a conservative design since the "uncertainty 
signal" is assumed to act in the most unfavorable manner. Bounded 
input - bounded output stability is guaranteed, provided the transfer 
function is of minimum-phase type. If the uncertainties are bounded. 
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Figure 2.6 - Error Between System Output and Model Output for 
Different Values of R 


Figure 2.7 - Characteristics of Higher Order State x 2 of the 
System for Different Values of R 
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Figure 2.8 - Characteristics of Higher Order State x 3 of the System 
for Different Values of R 


u(t) 
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Figure 2.9 - Control Signal Characteristics for Different Values of R 
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it has keen shown that the system can always be stabilized if suffi- 
cient control amplitude is available . These results also apply for 
systems with rather general nonlinearities that do not involve the 
control. It has also been shown that the tracking error admits an 
upper bound and that the bound can be made arbitrarily small. Since 
very little a priori knowledge about the system is assumed, the feed- 
back controller may be forced to supply excessive feedback gain in 
order to insure stability. This is, it seems, a logical compromise 
under the present situation. It is also clear that the controller 
uses partial state feedback from the states of a certain companion 
form. In practice, it is possible that some of these state variables 
may not be available. This requires reconstruction of the states from 
the available states or output. State reconstruction via well known 
Luenberger observer is difficult, because of the uncertainty in the 
system parameters. The problem may be compounded by measurement noise. 
Differentiation of the output to provide unavailable states together 
with a high gain design is likely to yield unsatisfactory performance 
in the presence of measurement noise. These problems will be tackled 
in the next chapter. 
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III DESIGN OF REDUCED ORDER COMPENSATOR AND 
ESTIMATOR FOR SYSTEM WITH 
PARAMETER UNCERTAINTY 

3.1 INTRODUCTION 

In the previous chapter, it has been shown that a linear time- 
invariant controller can be designed to stabilize a single input - 
single output nonlinear system provided that the system output and 
its lower order derivatives up to (n-m-1) are available. Here n is 
the system order and m is the number of zeros. In many practical 
situations, these state variables may not be available. In addi- 
tion, output measurements may also be noisy. The inherent diffi- 
culty associated with differentiation and possibly high gains limits 
the minimax procedure described in previous chapter. 

This chapter will deal initially with the generation of the re- 
quired control signal as the response of a dynamic system to the 
available states or output for deterministic case. The dimension of 
dynamic compensator is arbitrary. Because of the previous theoreti- 
cal development in Chapter 2, the highest order dynamic compensator 
will be (n-m-1). The problem is to design a dynamic compensator 
which is in some sense best to generate the required control signal. 

It is shown that the constant parameters that specify the dynamic 
compensator can be obtained by solving a set of simultaneous nonlinear 
algebraic equations. 
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The second problem in this chapter deals with the estimation of 
states of a dynamical system, given noise-corrupted observation, when 
there is parameter uncertainty in the dynamical system. The deter- 
ministic case will be treated first. 

3.2 DETERMINISTIC CASE: DESIGN OF DYNAMIC COMPENSATOR 

One way of estimating the states or generating an optimal control 
would be to assume that the observation is contaminated with noise 
and then use a usual Kalman filter. In addition to the dimension- 
ality problem of Kalman filter, it is difficult to specify various 
covariance matrices of plant disturbances and measurement noise in an 
essentially deterministic situation. Thus an alternative approach of 
generating the optimal control will be suggested below. It is to be 
noted that the gains of the dynamic compensator should be independent 
of initial state of plant and compensator, otherwise the compensator 
gains will have to be changed with the change of plant state due to a 
disturbance. 

3.2.1 PROBLEM FORMULATION 

The model reference system described in previous chapter is shown 
to be described by 

e = A^e + Su + jj£ (3.1) 

1 =C ® (3.2). 

where £ is r-dimensional output vector . 

The dynamic compensator of specified order, s, is described by 

z(t) = Fz(t) + GC(t) = Fz(t) + GCeCt) (3.3) 

u(t) = hz(t ) + hC(t) = hz(t) + nCeCt) C3.4). 

The input to the dynamic compensator is the available output whereas 
its output is the required control signal. It is easy to see that 
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C is r x n-m matrix 
F is s x s matrix 
G is s x r matrix 
h is 1 x s vector 
n is 1 x r vector 

The present formulation of the problem is similar to the one re- 

F251 

ported by Johnson and Athans L 

To design via optimization, the following cost function is chosen: 




J(r,G,h,n;5) * J 


{e T Qe_ + £ T (g T R 2 g + n^n)^ 


t<3 + z T (F T R 2 F + h^h)* - L£ 2 ] dt 


(3.5) 

The problem is to minimize the above criterion with respect to 
F, G, h, n and maximize w.r.t. £ , subject to (3.3) and 


e = A Q e + 0u + 6£ 

= (Aq +8nc)e^ +8hz + B£ (3.6) 

The inclusion of the second and third terms in the performance cri- 
terion avoids placing the poles of the compensator at -• and thus 
allow the high frequency plant noise not to pass through the system 
as mentioned in C25]. 

Now defining 


A 

n h 

A 

A O' 

: a 

£ °‘ 


* A 

'C 

o' 



P = 


, A = 


, 6 = 



, c = 







G F^ 


0 0^ 


0 I 




0 

I 

- 


~ A 

I 0 


'Q o 

a 

*1 c 

1 

A 

€ 

■ 

* 

' £ ' 

T = 


ii 

O' 

A 


, R = 



, -w = 



, 6 = 



0 I 


0 0 


1° r 2 ] 


2 

1 



0 
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it can be easily seen that (3.3) and (3.6) can be represented as 


m 

e 


A + 8nC 

8h 

|e 1 



w = 


= 



I 

+ 



z 


GC 

F 

J 

\z 


0 


- * 

i.e. w = (A + 6PC)w + £ 5 (3.7) 

and (3.5) can be written as 

J(P,5, w(to)) = j | f [w T Qw - L£ 2 ] dt (3.8) 

t 

o 


^ ^ i ^ flri iti ^ ^ ^ ^ ^ 

where Q = Q + - (C P RPC + TC RPCT) 


(3.9) 


Thus the problem is to minimize and maximize (3.8 ) with respect to 
P and £ subject to (3.7). The resulting optimal P will depend on 
initial condition w(to). In order to remove this restriction w(to) 
can be treated as a random vector in which case J may be replaced by 


J(P,£) = E[J(P,£, w( to) 3 • 


[2D 

Thus the requirement for optimal solution yields 
r 

~fp = Ip e £ j < p ’S» "(to)] = E[ f]F (P,5» w(to)] = 0 (3.10) 

A 

3J = 0 

3£ (3.11) 

The interchange of order of expectation (i.e. , integration) and 

differentiation is crucial here and is valid under rather general 
CLO] 

conditions 


3.2.2 ANALYSIS 

Applying the technique presented in Chapter 4, it can be easily 
seen that the optimal gain matrix P is given by 



44 


P = R -1 b T KLC T [j C(L+TLT)C T 3 


>1 


(3.12) 


where 


M = E[ J* (w w T )dt] = j*%(t) E[w(to)w T (to)]i|> T (t)dt 


i.e., - M = (A+8PC)M+M(A+BPC) T +E[w(to)w?to)] 
and K satisfies 

-K = (A+3PC) T K+K(A+BPC)+Q-k|l -1 I T K 


(3.13) 


(3.14) 


It t^-*«> , t -K), M and K are the steady state solutions of (3.13) and 
1 o 

(3.14) respectively. 

If 


e(o) 

z(o) 


= E£w(to)] = .wo 


and E[(w(to) - vr)(w(to) - = W Q 


Then E[w(to)w T (to)] = W + vv T 

o — o 

The optimal cost can be seen to be 


(3.15) 


J = - Tr (KE[w(to)w T (to)]} = - Tr {K(w + ww. T )) (3.16) 

2 2 o —o—o 

Thus the optimal signal requires only the available states ce and 
the states of dynamic compensator. This control signal is now ap- 
plied to the original system (2.9). Overall stability of the sys- 
tem is not apparent and is the subject of future investigation. 
Computational schemes to solve the simultaneous nonlinear equations 
have been discussed in Chapter 4. 


3.3 STOCHASTIC CASE: DESIGN OF ESTIMATOR 
3.3.1 PROBLEM FORMULATION 


It is assumed that a continuous record of a realization of the 
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(n-m)xl vector observable process {z^(t)> is available where 

z(t) = Hx(t) - Hy(t) + n(t) = He(t) + n(t) (3.17) 

and n(t) is white noise. 

Note that this means that the model is used in the estimation 
process to generate z(t). Moreover, the process {e^} assumed for 
mathematical treatment to be a random one, is modeled as the output 
of a dynamic system, excited by a pseudo-random signal £(t), uncor- 
related with n(t); that is, 

£ = A Q e + £(t) + £(t) 

g(t) = £u (3.18) 

where £(t) and n(t) are assumed to be zero mean, random processes 
having covariance matrices Q and R respectively. In case, £(t) has 
a bias, the system equation (3.18) can be augmented to take into 
account this factor. 

The problem is to determine the initial state < 2 (to) and £(t) for 
all [t 0 ,t f ] which minimize 


1 

2 


[(z- H e) T Ri" 1 (z-He) + + 

\ [e(to)-<5(to)] T P 1 [£(tb)-£ (to)] 


(3.19) 


subject to (3.18). 

& 

e(to) is the expected value of e(to). It is well known that, if 
£(t) and n(t) are sample functions of white, zero-mean, uncorrelated 
random processes, this procedure will give a maximum a posteriori 
estimate. 'If, in addition, C(t) and n(t) are gaussian, this will 
give a least mean-square error estimate. Even if C(t) is not ran- 
dom, the above performance index carries meaning in the sense that 
an integral square error in the estimates is being minimized. Since 



z(t) and He(t) are the actual observation and predicted observation 
respectively, [z(t)-He(t)] is the error in the estimate and 
provides relative weighting (related to the covariance of the noise) 
on this component. 

3.3.2 ANALYSIS 

The estimate e(t^) of e(t) at t=t^ is the solution of equation 
(2.18) using e( to) and l(t) as estimated. To obtain e(to), JjXt), 
the usual Hamiltonian 

H = \ (zr H e) T R 1 ' 1 (3-He) + | £ T Q + x’iA^ + £(t) + ^(t)3 

(3.20) 

is introduced with the costate equations and necessary conditions 

X = - ~ = hV 1 (z-He) - A T X (3.21) 

(3.22) 

(3.23) 


XX to) = -P~ x Ce_(to) - e(to)] (3.24) 

X(t.) = 0 (3.25) 

— i • 

Using (3.22), (3.23) reduces to 

e_ = A^e_ = Q^X + g(t) . (3.26) 

Now we claim a feedback solution of the form 

X(t) = K(t) Ce(t) - v<t)j .~ 03.27) 


Using (3.21), (3.26) and (3.27), it is clear that K and v satisfy 
the following equations: 


0 ^ 




— = * + Q^ 1 = 0 


®. = A q®. + 


and the boundary conditions 
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-K = KA q + aJk - KQ^K + H T R 1 ~ 1 H (3.28) 

v = A Q v - K _1 H T R 1 " 1 [z-Hv] + g(t) • (3.29) 

Again as in (3.25), 

X(t f ) = K(t f ) Ce(t f ) - v(t f )] = 0 . (3.30) 

Thus ~ 

e(t_) = v(t_) is the estimate e(t_) of e(t) at t_ given 

— r — r — r — r 

z(t) on [t ,t ]. 

— or 

Also from (3.24) and (3.27) 

i * 

K(to) = -P' 1 , v(to) = e(to) . (3.31) 

Thus 

e(tf/t f ) = v (t f ) = A Q e (t f /t f ) - K" 1 H T R 1 ” 1 Cz(t f )-He(t f /t f )] 

+ g(t) (3.32) 

where K is given by (3.28). 

Now defining 

M = -K -1 , 

it follows that ft = K ^KK 1 .. 

Thus the estimator is given by 

e (t f /t f ) = A Q e_ (t f /t f ) + MH T R 1 “ 1 C£(t f )-He(t f /t f :)3+ g{t) 

a it 

e_ (to/to) = e(to) (3.33) 

where the positive definite matrix M satisfies 

M = A Q M + MaJ + Q - MH^*R 1 ~^HM (3.34) 


M(to) = P(to) 


(3.35) 
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3.4 EXAMPLE 

Case 1 : Deterministic Case 

Following example will illustrate the design of a first order 
dynamic compensator for a third order nonlinear system described by 

(3) (2) * ' 

x + x + 2x - x - smx = u + u (3.36) 

with a second order model 
( 2 ) 

y v ' + 2y + y = r . (3.37) 

A second order model is chosen since the difference between 
plant order and the number of zeros is two. (3.36) and (3.37) can 
be combined to yield the error equation 

e (2) +2e+e=u+5 (3.38) 

u = u - r . 

It is clear from the previous chapter that the optimal controller 
requires measurements of e and e. It will be assumed that only e is 
available. Next a dynamic compensator 

z = fz + ge (3.39) 

is defined. The output of dynamic compensator is the required control 
signal given by 

u = hz + ne (3.40) 


With Q = 


1 0 
0 1 


= .01 , R 2 = .015 and L = .01 


(3.12) - (3.14) are solved using the algorithms reported in next chapter 
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to obtain 

f=-ll. 68 g= 8.2 h=-51.2 n=-47.68. 


The control signal is then applied to the original system (3.36). The 
outputs of model and the plant are shown in Figure 3.1. The control signal 
characteristic is shown in Figure 3.2. Thus the example establishes the 
fact that a reduced order dynamic compensator can be effectively used to 
generate the control when some of the necessary states of certain com- 
panion form are not available. 

Case 2: Stochastic Case 

In this example, a first order estimator will be designed for the 
above model-reference system to estimate e and e from the observation 
z(t) = e(t) + n(t) 

= he(t) + n(t) (3.41) 


h = (1 0), 

where n(t) is a gaussian noise with standard deviation .01. 
10 0 


Using = 


55.36 


= 1, the estimator is given by 


A 

-24.56 1 

A 

24.56 


0 

e_(t f ) = 

1.32 -2 

e(t f ) + 

-2.36 

z(t) + 

1 


u 


(3.42) 


The controller is designed using the procedure of the previous 


chapter with Q = 


, R=L=0.01 and is given by 


u = -R -1 ( .5, . 5)e . (3.43) 

The control signal is then applied to the system (3.36). The outputs 
of the plant and model are shewn in Figure (3.3). Noise corrupted ob- 

A 

servation z(t) and its estimate e(t) are shown in Figure 3.4. The con- 
trol signal is plotted in Figure 3.5. Thus a reduced order estimator can 
be designed to implement the controller for a class of model reference 


system. 
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LCS 
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Figure 3,4 Moise corrupted observation z and its estimate 
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3.5 CONCLUSION 

In this chapter, the problem of generating the optimal control as 
the output of a dynamic compensator is treated for system with para- 
meter uncertainty. The input to the dynamic compensator is the 
available output of the system, whereas its output is the required 
control signal. The minimax technique of the previous chapter has 
been extended to the case where the state variables are not in phase 
variable form and also the necessary state variables are not avail- 
able. 

Finally an ad hoc scheme for estimating the necessary states of 
the system with process uncertainty has been developed using a re- 
duced order deterministic model. Very little a priori knowledge of 
the parameters is assumed. If the signal related to the uncertainty 
and the measurement noise are uncorrelated, the resulting estimator 
is linear and is optimal in the sense that it minimizes a quadratic 
criterion involving estimation error and a signal related to uncer- 
tainty. The minimax technique presented in Chapter 2 and 3 has 
some limitations. First, it is difficult to extend this basic con- 
cept directly to general multivariable system. The difficulty is 
due to the fact that a suitable canonical form for multivariable 
case is not available. Second, the uncertainty signal £(t) is re- 
lated to the system parameters in a complicated way. Thus an ulti- 
mate bound on parameter variation to insure system stability is 
difficult to ascertain. The problem of controlling multivariable 
system with parameter uncertainty will be reformulated and will be 
treated in the next chapter. 
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IV MINIMAX OUTPUT FEEDBACK CONTROLLER 
4.1 INTRODUCTION 

The design of a controller for a linear multivariable system 
having parameter uncertainty is explored in this chapter. Linear 
output feedback is employed with the feedback gains determined by 
minimizing one of several criteria. The problem is treated initially 
by minimizing with respect to the feedback gain matrix and maximizing 
with respect to uncertainty, a quadratic performance index involving 

r 9 l 

the system state, the control and the "uncertainty signal" 

The optimal gain matrix satisfies a set of simultaneous nonlinear 
algebraic equations. The design procedure often leads to a pessi- 
mistic result, either because the uncertainty does not act as per- 
versely as assumed, or because the control often makes an effort to 
reduce the cost where it is high, even with perfect knowledge of 
parameter. To meet this objection, other criterion and in parti- 
cular, a minimax sensitivity criterion^^ are also examined. The 
optimal feedback gain matrix for the so-called "regret criterion" 
is shown to satisfy a set of nonlinear equations similar to those 
obtained for the standard criterion. It is demonstrated that various 
minimax design criteria yield better system performance under wide 
range of parameter variation that a purely nominal design. 
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4.2 SYSTEM DESCRIPTION AND PROBLEM FORMULATION 

Consider an n tl1 order linear system with state vector x(t)eR 

— m 

and output vector y(t)eRq defined by 

x = A^ + BJJ + (A-A q )x + (B-E 0 )u (4.1) 

y = Cx (4.2) 

with a controller 


u = -Fy = -FCx . 


(4.3) 


where A^.B^ are nominal matrices. Using (4.3), (4.1) can be 
represented as 

x = (A 0 -B 0 FC)x + [(A-A 0 )-(B-B 0 )FC)x 

= (A 0 -B Q FC)x+ (W-W 0 )x = (A 0 -B Q FC)x +D£ (4.4) 

where £ represents the effect of uncertainty. 

Since the uncertainty is assumed to be limited , £ will likewise be 
constrained. In order to place any restriction on the form of (W-W Q ) 
let W-Wq = DGC^. D, C^ are fixed and G contains variable terms. An 
example is 

TOO 0 1 


B = B q , A-A q = 


0 0 
^ a o _a o^ * ••• 


'Vi-Vi' 


= I, G = [Qq^oIq , *** 


Thus the uncertainty vector is specified as 

£ = GC x x (4.5) 

where G is the gain matrix associated with the uncertainty vector 
and C^ has rank n or less. Both C, C^ .are assumed to have maximum 
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rank, i.e., rank equal to number of rows. 

Substitution of (4.5) in (4.4) gives 

x = (A q -B 0 FC + DGC 1 )x. (4.6) 

In order to achieve a design through optimization, the feedback 
matrices F and G will initially be chosen to minimize and maximize, 
respectively, the performance criterion 


J(F,G) = 


1 

2 


[x T Qx + u T Ru - 5 T L£] dt 


u=FCx , £=GCx 


= J 0 (i»£) 


u=FCx, £=GC-.x 


1 

2 


x T Q + C T F T RFC - C^G T LGC 1 ] x dt; 


(4.7) 


i.e. 


(i) Find F and G such that 

min max J(F,G) = max min J(F,G) , (4.8a) 

F G G F 


We shall also consider the following minimax procedures to obtain 

optimal gain matrix F ; 

(ii) min J (u,£*)| 

u 'u=FCx 

— — fco 

1 T 

where £* is obtained from max [min J (u*, 0) - — dt] 

£* U* 0 2 J Q 

and vi* minimizes J Q assuming G=0. (4.8b) 


(iii) min max [J(F,G) - J*(G)] (4.8c) 

F G 1 

where J*(G) = min [J(F,G)] given G. 

1 F 
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(iv) min max [J(F,G) - J*(G)] (4.8d) 

F G 2 

where J*(G) = min [Jq(u 

2 u £. = ^C|j< 

Criterion (ii) is less pessimistic in the sense that £ is given the 
first play and, in making its play, assumes that u(x) is obtained by 
an optimal full state design (with £=0) for the nominal plant. 

Matrix F is then chosen to minimize the criterion based on the an- 
nounced strategy of £. 

In criterion (iii), (iv) the best control with perfect parameter 
information , i . e . , £ known , is obtained with output feedback and 
full state feedback respectively. Matrices F, G then minimize and 
maximize respectively the difference between the actual cost and cost 
with perfect parameter information. 

4.3 MINIMAX PERFORMANCE CONTROL WITH DIRECT CONFLICT OF INTEREST 

In this case, the saddle point is defined by the following inequal- 
ity 

J(F*,G) £ J(F*,G*) £j(F,G*) . (4.9) 

It is clear from (4.6) and (4.8) that J is determined by the initial 
state, x(tg) as well as matrices F and G. That is, 

J = J(F,G,x (t Q )). (4.10) 

In order to make the optimum F and G independent of x(t Q ), x.(t Q ) can 
be treated as a random vector in which case J may be replaced by 

A 

J(F ,G) = E[J(F,G,x(t 0 )]. (4.11) 

E (•) denotes expectation with respect to x(t Q ). The necessary 
condition that F and G should minimize and maximize J(F,G), respec- 
tively, requires 
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3J 3 3iT 

3F = 9F E CJ(F,G,x(t 0 ))] = E[^|r (F,G,x(t o ))] = o ( 4<12a ) 

3G = 3G = E ^3G^ F,G> — ' = ° (4.12b) 


The interchange of order of expectation (i.e., integration) and 
differentiation is critical here and is valid under rather general 
conditions ^ * 

The partial derivatives of (_4.12)will be evaluated by the appli- 
cation of following Lemma: 

Lemma 4.1 


If 


J=J(x(t Q ))= W(j4(t Q )) + 


L(x,t) dt , 


where 


x = f(x,t) and W(x(tg)) is the penalty on the initial 
states x(t Q ), then 
3J 




3Xi(t 0 ) " i *0 ' 9x i 


where 



h CL + = o . 


This Lemma follows from the variational calculus where the first 
variation of J with respect to x(t Q ) is [X_(t Q ) + 9W/3x] T <5_x(t 0 )f 13 -^ 
In order to apply the lemma, the elements of F and G are treated as 
additional "states” which satisfy 

F = 0, 6 = 0. (4.13) 

Vector multiplier will be used for the regular state constraint 
and matrix multipliers Ap(t) and Ag(t) will be used for matrices 
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F and G respectively. It is to be noted that the Hamiltonian H will 
be independent of Ap(t) and Ag(t) due to (4.13). Thus the Hamiltonian 
H for (4.6), (4.7) is 


H = ix C(A 0 _B 0 FC 0 + DGC i ) 2i ] + ^ + cTfTrFC ~ C^G T LGC 1 ]x 


J . 1 


= Tr[CA 0 -B 0 FC 0 + DGC-^x + j (Q + C T F T RFC - C^G'lGC^joc 1 '] 


•>T^T r 


(4.14) 


Tr denotes the trace and 
3 h 


-iTr.Trjr-/-, oTr-T. 


X* = - = - (Aq-BqFC + DGCj^X^ - (Q + C'F 1 RFC-C 1 G 1 LGC)x,X x (t f )=Oj 

(4.15) 

(4.16) 


A F (t) = = - RFC^V + B^xV, ^.(tf) = 0 ; 

• T T T T T 

V »> * - 3G 1 ^lS c i - » v t f ) = °- 


(4.17) 


According to the lemma, the necessary condition (4.12), and integrated 
forms of (4.16) and (4.17), we obtain 

ft. 


0 = EC|£] = E[A F (t Q )] = E 


T T T T T 

r [RFCxx C - B X x C ] dt 
— cr-x— 


(4.18) 


0 = EC|i] = ECA G (t 0 )3 = E 


ft 


f T T T T T 

[LGC^xx - DAxC|] dt . (4.19) 


Thus if R and L are constants, (4.18) and (4.19) yield 
-1 


F = R 


| f B 0 e ^ k x T 3C T dt I J f C E[xx 


XT -1 

]C dt J X ; (4.20) 


G = L 


-1 


f f t T T 

D E[X x ] C dt I 
-x- 1 


f C 1 E[xx T ]C* dt J -1 . (4.21) 
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(4.20) and (4.21) can now be simplified. If X = K(t)x Is assumed, 
then(4.6) and (4.15) give 

-K= A*K + KA* + Q + C T F T RFC - C^LGC^ K(t f ) = 0 (4.22) 

ft W T T m . 

or K(t) = I f 4> ft (t,t) [Q + C F RFC - cWbGCj] 4 A (T,t) dt 


(4.23) 


where <p A is the transition matrix corresponding to A ft = (A q -BqFC + 
DGCj^). Limiting attention to the time invariant case (Q,A 0 ,B 0 ,C,D, 
constant) with t^ =a> , tQ=0 , equations (4.20) - (4.23) yield 


-1 T T T -1 
F = R B^KMC [CMC ] X 

G = L _1 D T KMC^ [C^MC*] -1 


(4.24) 

(4.25) 


where 


K = 


f 00 Ai(t-t) T T _ A (t-t) 

e (Q+C^ 1 RFC - CVLGC^ e dr 


°° A *° T T T T A .o 

e (Q t C 1 F 1 RFC - C^ LGC 1 ) e do (4.26a) 


or 


K(A -B FC + DGC ) + (A-.-B_.FC + DGC, ) T K + Q + C T F T RFC - 
0 0 1 0 0 1 

T T 

C.G LGC. = 0 
11 » 


(4.26b) 


and 


M = 


Efxx 1 3 dt = 


' ” A * t T A I t 

e E tx(t 0 )x T (t 0 )3 e dt. 


(4.27a) 


or 


(A.-B FC + DGC.) M + M(A -B n FC + DGC, ) T + E[x(t n )x T (t )] = 0. 
uo 0 u — u — 0 


(4.27b) 



63 


[CMC 


T l_ -2, 

3“ and [C,MCT] exist because C,C, have maximum rank and M is 
1 1 1 


positive definite. 

[x(t )3 = Xq » 1-q / ilQ y 


If E[x(t^)3 = x n , E[x(t^)-x^) (xft^-Xn) 7 ] = X Q 


(4.28) 


then 


E C*( t 0 )* T ( t o ) 3 = X Q + 2So2So 


(4.29) 


is positive definite for X^^O. Thus M is a positive definite solu- 
tion of (4.27b) if Xq^O, M is positive semi-definite if X Q =0. 

The optimal cost can be seen to satisfy 


J = ~ E C x T (t Q ) Kx( t Q ) ] = j Tr [KE(x(t 0 )x T (t 0 ))3 = 

= ^ Tr [K(X q +XqX^)] = i TR [K] when E [x(t 0 )x T (t 0 )3=I. 


Remark 4.1 


It can be easily seen that rain max J(F,G) 

F G 


max min J(F,G) 
G F 


4.4 COMPUTATION OF F* AND G* 

The feedback gain matrices F and G are specified by (4.24) and 
(4.25), where K and M are given by (4.26b) and (4.27b) respectively. 
These equations roust be solved numerically and the following algor- 
ithm similar to that presented in [34] can be conveniently used for 
this purpose. 

F , , G , and M , are computed by simultaneous solution of the 
n+1’ n+1 n+1 r 

following equations : 


i T T T -1 

F . = R" B'K C (CM.C ) 1 
n+1 0 n+1 n+1 


(4.30) 
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G n*l = ''iWl*' 1 ( "- 31 > 

<V¥»1 C + D1 Wl> «W + Vi< VVn + i c + 

D G n -j-i C 1 )T + 1 = 0 (4.32) 

where K n+ ^ is given by the following equation: 

("O-Vn 0 + >W * >W VoWl> 

+ Q + C T F T RF C - C?G T L6„C, =0. (4.33) 

n 11 x n “ -*■ 

Observe that (4.33) is approximate, while (4.30) - (4.32) are exact. 
The iteration starts with an initial guess of Fq and G Q such that 
(A 0 -B 0 F q C + DG 0 C 1 ) is stable and also (Q + C T fJfR 0 C - C^GqLG q C) is 
positive definite. Then is the positive definite solution of 
(4.33). With this value of K^, (4.30) - (4.32) can be solved simul- 
taneously to get F^, G^, which, in turn, give new estimate, Kj, 
and the iteration proceeds. Alternately (4.24) - (4.27) can be 
solved simultaneously. 

Lemma 4,2| 

If (BqR'^Bq - DL ^D^) >_ 0 and C = C^, the above algorithm will 
converge in the sense that TrCKjj-K^^] >_ 0 for all n. 

Proof: 

The proof closely follows [3*0. 

can be expressed as 

( “ t a T 

M = 6 . dt = ^ 

n J *n *n n n 
0 


(4.34) 
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If C = C^, then 

% 

(B Q F -DG ) C= (BoR'^J-DL'V) KM C T (CM C T ) -1 C (4.35) 

w ** n u o nn n 

and 

C T (F T RF G T LG )C = C T ( CM C T ) _1 CM K (B R~ 1 bI'-DL" 1 D T ) K M 
n n— nn n nnOO nn 

C T (CM cV^C (4.36) 

n 

Substitution of (4.35) and (4.36) into (4.32) and (4.33) yield 
equations identical to those of [34] for which Tr[K -K^^] ^ 0 is 
proven except that B Q R -1 B^-DL 1 D T replaces BqR'^Bq.. Thus Lemma 4.1 
holds. Proof of convergence under less restrictive assumptions is 
the subject of further investigation. 


4.5 MINIMAX PERFORMANCE CONTROL WITH INDIRECT CONFLICT OF INTEREST 


In the previous formulation, the feedback matrix F has been chosen 
in a most favorable way after the uncertainty vector was allowed to 
take its "worst" value. This will lead to a very conservative design 
approach. On the other hand, it may be assumed that nature is not 
perverse enough to alter its strategy with that of the control. 

Under this situation of indirect conflict of interest, the previous 
formulation may be modified as follows. 

The game is, as usual, defined by 

x = AqX + BqU + D^_ . (4.37) 

To start with, let us assume £=0. The optimal control u* is obtained 
by minimizing r 00 


J 



(x 7 Qj< + u_ T Ru) dt 


(4.38) 


subject to (4.37). 
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Thus the resulting control is given by 

u* = - R~ 1 B T P_x 
— 0 n 0— 


(4.39) 


where is given by 

* P 0 A 0 ♦ « - p o b o r ' 1b J p o 

Substitution of (4.39) in (4.37) yields 

i. * < A o- B o R ' lB o p o>i ♦ D I • 


= 0 


(4.40) 


(4.41) 


To limit the uncertainty at this stage, the performance criterion 
(4.38) is modified as 


J 


1 

2 


* 00 

Cx T (Q+ p 0 B 0 R' 1b JPo)x - £ T L£] dt 

0 


(4.42 ) 


The 'worst* value of £ is obtained by maximizing (4.42) with respect 

to £, subject to (4.41), and is given by 

= L“ 1 D T P.x (4.43) 

-0 1— 

where P^ is the solution of 

( VV' 1b o p </ p i + VW'^V + <5 + p o b o r ' 1 b S p o 

-P 1 DL _1 D T P 1 =0 (4.44) 

Using the estimate of £_ as in (4.43), the original system is reduced 
to 


x = (A + DL -1 D T P, )x + B.u 

- 0 1 — o- 

with the controller 
u_ = ■‘Fy = -FCx 

Now F can be chosen to minimize E[J] = y E 
subject to (4.45). The optimal F is given by 


(4.45) 


(4.46) 

► CO 

x T [Q+C T F t RFC]x dt 

. 

0 
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F = -R _1 B T PMC T ( CMC 1 ) -1 (4.47) 

0 

where P and M are given by 

(A 0 + dl' 1 d t p 1 - B q FC) T P + P (a 0 + dl' 1 d t p 1 -b 0 fc) 

+ Q + C T F T RFC=0 (4.48) 

(A q +DL" 1 D T P 1 -B 0 FC)M + M(A 0 + DL' 1 D T P 1 - B 0 FC) T +I=0 (4.49) 


Remark 4.2 

(a) To be more general, u* in (4.31) and in (4.43) may be 

0 — 

constrained to the form 

h§ = » si = g o c i^ 


(b ) It should be noted that this formulation assumes the exis- 
tence of matrices P Q , P^ and F that stabilize (A Q +DL 1 D^P^-BqFC) 
and (A 0 -B 0 R -1 B 0 t P 0 -DL -1 D t P 1 ). Under this condition, (4.47) - (4.49) 
Ccm be solved using basically the same algorithm as described in 
section 4.4. 

4.6 MINIMAX SENSITIVITY (OR LOSS) CONTROL 


If G as defined in (4.5) were known, the ideal optimal control 
would be obtained by minimizing 


f 00 


J = 


{x T (Q-C^G T LGC 1 )x + u T Ru} dt 


(4.50) 


with respect to F^ subject to 


x = (A Q + DGC^x + Bu . 


(4.51) 


where 


u = 

The resulting optimal F* is given by 

A ■. rnilt t if rp , 

F^ = r - 1 bJkmc t (cmc 1 ) x 


(4.52) 
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* * 

where K and M satisfy 


N-^fJ.G.K) = (A Q + DGC 1 -B 0 F*C) T K + K(A Q + DGCj-BqF^C) 


+ q + c t f* t rf*c - C^G T LGC 1 = 0 


(4.53) 


N„(F*,G,M) = (A n + DGC. -B rt F*C)M + M(A rt +DGC. -B n F*C) T +I=0 

* x u i.0j_ o -l o i 

(4.54) 

The ideal optimal control using output feedback is then 


a is 

u = -F^Cx 


ft 


where x satisfies 


(4.55) 


* , * * 
x = (A + DGC X - BF^Ox 


(4.56) 


and the 


& 

resulting cost J (G) = min J is given by 
1 T, 


• I 


*T T *T * T T * 

x 1 [Q + C F^RF-jC - C^LGC^x dt . (4.57) 


This is the best that can be achieved with constrained feedback 
(4.55) and perfect parameter information (G) . 

Note that 

E[x(to)x (to)] = E[x(to)jc T (to)] = I (4.58) 

has been assumed in (4.54) . 

Now the following performance sensitivity or "regret loss" criterion 
is considered: 

S(F,G) = f[J(F,G),J*(G)] (4.59) 

Definition 4.1 

S(F,G) as defined in (4.59) is a performance sensitivity function 


if [ 11 j 54, 56] 



69 


1) f ( * ) is continuous jointly in its two arguments 

2) f>0 J(F,G) > J*(G) 

3) f=0 -*■ J (F,G) = J*(G) 

1 

In this thesis, attention has been confined to the following 
sensitivity function 

S(F,G) = J(F,G) -J*(G) (4.60) 

A 

The immediate problem is to minimize and maximize S with respect 
to F and G respectively, subject to (4.6), (4.56), (4.53) and (4.54). 
(4.60) modified to include the equality constraints (4.53) and (4.54) 
is 

S = Tr [N 1 (F*,G,K)P 1 + N 2 (F*,G,M)P 2 ] + 

| x T [Q + C T F T RFC-C 1 T G T LGC 1 ]_x dt 
0 

1 t °° *T «i« A T T * 

- “E x [Q + (TF, RF.C - cjG^GC.Dx dt 

2 J ~ 1 1 1 X ~ (4.61) 

0 

where P^ and P 2 are matrix Lagrange multipliers. 

Thus the problem reduces to minimizing and maximizing (4.61) with 

respect to F and G respectively, subject to (4.6), (4.56) and 

* 

f=0,G=0,F i =0 (4.62) 

The Hamiltonian H for this case is given by 

H = j Tr [(Q + C T F T RFC - C^LGC^x x T ] - j Tr [(Q + C T F* T RF*C - 

C T G T LGC.)x x T ] + Tr [(A -B FC + DGC. )x X T ] + 

1 00 1 x 

Tr C( a 0 " B 0 F 1 C + 


(4.63) 
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with the ' costate * equations 


i =-!» = 

-x 3x 


- (Q + C T F T RFC 


T T * 
C 1 G i LGC 1 )x 


(A n -B FC + DGC,) A , 
0 0 1 -x 


X („) = 0 


(4.64) 


•* 

V 


3H 

J* 

x 


(Q + C T F^ T RF*C 


T T * * T« 

C 1 G LGC.)x - (A + DGC ,-B.F C) 1 X* , 
1 JL — 0 10 — x 


X.(eo) = O, 
x 


A_ = - 


8H . 
W ~ 


T T T T T 
RFC xx C + B* A x^C 1 , 
0— x~ 


(4.65) 


(«) 

A =0 (4.66) 

F 


• 3 H «r T ** T T T T t T 

A = - -£ = LGCxxC - LGC xxC - D A x C, - DA x Cl , 
G 3G 1 — 1 — 1 -x— 1 1 


V*> 


= 0 


A* 

F. 


3H * **T x T *T T 

= RF Cxx C + B X.x C 

3* i — 

r* X 


A.. ("> 
F i 


= 0 


(4.67) 

(4.68) 


Now according to Lemma 4.1 and the necessary conditions (4.12), it 
can be easily seen after integrating (4.66) - (4.68) from 0 to ® that 


0 = E[A f (0)J = E 


[RFCxx T C T - b\ x T C T ] dt 
— 0— x— 


(4.69) 


o = E[A g ( 0) + ~Tr (N 1 (F*,G,K) P 1 + N 2 (F*,G,M)P 2 } ] 

T* t T TTT t * T 

= D K (Pj+P^X^ - LGC 1 (P 1 +PpC 1 + D (P 2 + P 2 )MC 


- LG 


E( c 1 xx T C^) dt + LGC 1 




E(xx ) dt C 


= D 


X x T cT dt + D X 
-x- 1 


*T T 

A...x C 
-x~ 1 


(4.70) 


0 



71 


0 = E[A., ( 0 ) + i- Tr (N (F*,G,K) P + N_ (F*,G,M)P )3 
" B 1 1 1^12 


T* TT * TTT t A t 

= - B K (P +P)C + RF C(P + P;)C - B (P + P X )MC 
Oil 11 1 022 


A 

- RF. 


A At t t 

E(Cx x C ) dt - B 
0 


*T T 

E(X x 1 )C dt 


0 0 

As usual, the following feedback solution is assumed: 

-A 

X = Kx , X. = Kx 
“* “ x 


(4.71) 


(4.72) 


This can be verified by (4.64) and (4.65) to be valid provided K 
$ 

and K satisfy 

(A 0 -B 0 FC+DGC i ) T K+K(A 0 -B 0 FC+DGC 1 )+(Q+C T F T RFC-C^G T LGC 1 ) = 0 ? (4.73) 

(A 0 -B 0 F 1 C+DGC 1 ) T K+K(A 0 -B 0 F 1 C+DGC 1 )-(Q+C T F^RF 1 C-C^G T LGC 1 )=0 • (4.74) 


Furthermore 


M = 


E[xx T ] dt, M = 


A A>r 

E[x x ] dt 


(4.75) 


are given by 


(A o" B o FC+DGC i )M+M(A o" BFC+DGC i ) + 1 = 0 

(A -B FC+DGC. )M+M(A -B F.C+DGC.) T +1=0 

ooi i oof f 


(4.76) 

(4.77) 


It can be easily seen from (4.53), (4.54), (4.75) and (4.77) that 


K = -K, M = M 

Using (4.77), (4.72), and (4.75), (4.71) reduces to 


(4.78) 
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T* TT* TT T t~* T 

0 = -B q K ( P^+P^ )C 1 +RF^C (P X +P^)C - B J(P 2 +pJ)MC -RF^MC -B KMC 

(4.79) 

Using (4.52), (4.79) reduces to 

T* tT* TT t T * t 

0 = -B q K (P x +P*)C +FF c (P 1 +P 1 )C 1 - B* (P 2 +P 2 )MC X , (4.80) 

Now two cases may arise. 

Case 1. (P x + P 7 ) 0 

* * 

Solving for ., (4.80) yields 

F 1 = R ” 1b o C(P 2 +P 2^ + ^ P 1 +P 1 )3cTCC<P 1 +P 1 )cT:1 ’ 1, ( 4 - 81 > 

Comparing (4.81) with (4.52) yields 

(P 1 +P 1 ) = « 311(1 ( p 2 +p 2 ) = 0 (4.82) 

Using (4.82), (4.72) and (4.75), (4.69) and (4.70) give basically 
the same result as obtained via minimax performance control. 

Case 2. (P x + P 7 ) = 0 (4.83) 

Using (4.83), (4.80) gives 

(P o + P 7 ) = 0 (4.84) 

2 2 

Substitution of (4.83), (4.84), (4.72) and (4.75) into (4.69) and 
(4.70) gives 

F = R -1 bJ KMC 1 (CMC 7 )" 1 (4.85) 

-1 » ^ * t - " T -1 

G = L D X (KM+K M)C [C 1 (M-M)C ] # (4.86) 

Thus it is clear that soltuion of F requires simultaneous solution 
(4.85) - (4.86) together with (4.73) - (4.74) and (4.76) - (4.77). 
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Remark 4.3 

It can be easily verified that 

.V 

(a) The optimal cost S = j Tr (K + K) , E[x(to)xJ(to)3 = 

* svr 

E[x(to)x (to)] = I (4.87) 

^ a 

(b) min max S(F,G) = max min S(F,G) (4.88) 

F G G F 

£ 

It can be seen from (4.55) that u is constrained. A rather 
optimistic situation will be to allow u to have complete state 
feedback. The problem here is to minimize and maximize with respect 
to F and G respectively. The following sensitivity criterion 

S = J(F,G) - S 2 (G> (4.89) 

ft 

where J(F,G) is given by (4.7) and J 2 (G) is given by 

[x T (Q-C^G T LGC 1 )x - u T Ru] dt (4.90) 

0 

This is a special case of the constrained feedback problem and the 
required result is obtained by setting C=I in (4.55). Thus the 
required feedback matrices F and G are given by 


J.(G) = mm — 
1 2 
u * 


F = R -1 BqKMC T (CMC T ) -1 (4.91) 

G = L _1 D T (KM + KM)cT [C,(M-M)C^] -1 (4.92) 

- * . 

where K, K. , M, M are given by 


(A -B FC+DGC, ) T K + K(A ft -B n FC+DGC, )+Q+C T F T RFC- cTg T LGC 1 = 0 (4.93) 

001 0 0 1 l-l 

ft ft ft ft * * t t 

(A rt +B R _1 bL<+DGC ,) T K + K(A.+B n R" 1 B^K+DGC 1 )-(Q+KB R-Vk-CGLGC, ) = 0 
0 0 u 1 u U 0 q 0 1 

(4.94) 
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(A 0 -B q FC+DGC i )M + M(A 0 -B 0 FC+DGC 1 ) T +1=0, (4.95) 

* A A * 

(A-+B n R _;L BoK + DGC, )M + M(A rt +B R -1 B T K+DGC. ) T + 1 = 0 (4.96) 

uuu 1 OOo 1 

A 

4.7 COMPUTATION OF F, F 1 , G 

An algorithm similar to that mentioned earlier can be vised to 

solve for the feedback matrices. As before, at iteration n, positive 

* 

definite matrix and negative definite matrix K n are obtained from 
the following linearized equations: 


(A-BF C+DG C) T K -+K (A-BF C+DG C)+Q+C T F T RF C-C 7 G T LG C =0- (4.97) 

n n n+1 n+1 n n n n 1 n n 


a 

vT' 


* 


(A-BF^CtDG^)*^ + V 1 (A-Bf ln C + I*3 n Cp-(Q.C T r u Rr ln C-C^LG n C 1 )=0 

(4.98) 


A A 

F F. , G , M , M are then obtained by simultaneous solution of the 
n m n’ n n 

following nonlinear equations: 

* * A T 

F, = R" 1b 5„M C (C$ C T ) _1 (4.99) 

In u n n n 

<V DG n c l-Vla c) + 


-C^G T LGC 1 =0, (4.100) 

(A 0 +DG n C l- B 0^1 C) ^n + \ {A 0 +DG n C l" B 0*i C)T+I = °’ (4 * 101) 

* * 

F = R~ 1 b3’k,M C T (CM C 7 )' 1 (4.102) 

n 0 « n n 

A A A 

G = L -1 D T (K M + K M )c! [C 1 (M r ,-M r ,)C 7 ]" 1 (4.103) 

n • nn nnl inni 


The nonlinear equations (4.99) - (4.103) can be solved at each itera- 
tion n, by standard conjugate gradient technique. The algorithm 


starts as follows : 
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a) Assume initial guesses F 0» F 10» G q such that (Aq-BqFqC+DGqC) 
and (Aq-BqF^qC+DGqC) are stable and also (Q+^FqRFqC-C^GqLGqC) 

and (Q+C t F f q RF 10 C-C GqLGqC) are positive definite. 

b) Solve (4.97) and (4.96) and store the values K n+1 and K n+ ^ 

* 

c) Using K ntl and Kjj+i , solve (4.99) - (4.103) by conjugate 

it 

gradient technique to obtain F n+1 , F ln+1 , G n+1 , M n+1 , 
and Mh +1 . 

» A 

d) With these values, and are updated and the iteration 
continues till the specified stopping criterion is met. 

4.8 SOME STABILITY BOUNDS IN TERMS OF PARAMETER VARIATION 
The perturbed system (4.1) can be represented as 

x = AqX + BqII + (A-Aq)x + (B-B q )u (4.104) 

= [(A 0 -B 0 FC) + AA + ABFCjx (4.105) 

where F is given by (4.24). The following analysis is also true for 

F, given by (4.47), (4.35), and (4.31) . 

Define the Liapunov function V(x) as 

V(x) = -|x T Kx (4.106) 

where K, a positive definite matrix, satisfies (4.26). The time 
derivative V(x) of V(x), evaluated along the trajectory (4.105), is 
given by 

V(x) = - ^x T [-(A 0 -B 0 FC) T K-K(A 0 -B 0 FC)-2KAA-2KABFC]x (4.107) 
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Using (4.26), (4.107) reduces to 

V(x) = - |x T [(Q+C T F T RFC-C^G T LGC 1 )-2KM + 2K(DGCjK~ 1 + ABFCK -1 )K]x 

(4.108) 

Let the norms of x and matrix A are defined as follows 

I I I I ^ t T \^-/2 

llxll = <xx> 

f | A j | = sup | | Ax | | so that 1 1 A { | = [A^A] 

1 1 x J |=j. 1 — max 

where A ( • ) is the maximum eigenvalue of a symmetric positive 

IDaX 

definite matrix (*). Restricting terms in the bracket in (4.108) 
to be at least p.s.d. to guarantee stability of perturbed system 
(4.105), the bounds on AA and AB can be found as 

||aa|| < A . (Q + cV r RFC-C:fG T LGC 1 ) (4.109) 

II ' — mm 1 1 

2||K|| 

llA II < ll DGC 1 K ' 1 H 

. (4.110) 

i | fck _1 | | 

It should be noted that (Q + C^F^RFC-C^G^LGC^ ) is at least posi- 
tive semidefinite under the condition mentioned in Lemma 4.2 

4.9 EXAMPLE 

Following example will be considered to illustrate various theo- 
retical formulation discussed earlier. 

Let the system be described by 

u = Ax + (4.111) 
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y = [0 1] x = cx (4,112) 

with controller 

u = -fy = - F [0 1] x (4.113) 


'a' in (4.11) is the uncertain parameter. Let the nominal system 
correspond to the one with a = 0. Thus (4.111) can be written as 


- 

— 


r ■« 



0 

1 

X + 

0 

u + 

0 

-1 

O 

k 


1 




= AqX + bgU + d_ £ 


(4.114) 


with £ constrained to be 

£ = gx 2 = gy = gcx (4.115) 

where g is the gain (i.e., an estimate of the uncertainty) to be 
determined. 

Consider the following performance criterion 


with 


J = min max 
f g 


- 

“ 

E 1/2 

[ x^Qx+Ru^-L£^]dt 

► * 

0 


Q = 


"l 0 
0 0 


,R = 1, E[x(0)x T (0)] = I 


(4.116) 


f optimal for the nominal system (i.e., with no parameter uncertainty) 
is determined to be 0.816. f and g for different values of L are 
obtained through minimax procedures (i), (ii) and (iii) and using 
algorithms of sections IV and VII. Simultaneous nonlinear algebraic 
equations, e.g. , equations (4.30) - (4.32) of minimax procedure (i) 
and (ii) have been solved at each iteration using a conjugate gradient 
technique. The computed values of f for different values of L are 
tabulated for various minimax procedures. 
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Table 4.1 


— 

f 

-1 

L 

Minimax Performance Control 

Minimax Sensitivity Control 
Criterion (iii) 

Criterion (i) 

Criterion (ii) 

0.1 

.878 

.914 

.824 

0.2 

.947 

.96 

.844 

0.3 

1.03 

1.1 

1.08 

0.5 

1.265 

1.277 

1.354 

0.7 

1.69 

1.71 

1.815 


To study the effect of uncertainty, J is computed for different 
values of ’a’ using f as tabulated above and 

i 00 


1 1 

J = - Tr K = 7 E 
2 z 


[x T Qx + Rf 2 x 2 ]dt 


(4.117) 


where K is the solution of 


(A-b Q fc) T K+K(A-b 0 fc)+Q+c T f 2 c =0 (4.118) 

and are plotted as shown in Figures (4.1) - (4.2). In Figure (4.1), 
cost J is plotted as a function of the uncertain parameter 'a*, 
us ing the feedback gain as determined in minimax performance sensi- 
tivity criterion (N), for difference values of L. For comparison, 
we have also plotted the 'optimal* cost as a function of parameter 
'a* if it were known. In Figure 2, different design criterion are 
compared as "a" varies from nominal. It can be seen that the mini- 
max procedure effects the design of f in such a way that the system 
will operate acceptably over a wider range of parameters than a 
purely nominal design. 












Minimax Performance Sensitivity Cost Criterion (iii) 



Figure 4.1 



Minimax Performance Sensitivi 
Criterion ( ) R?1.0, JL _I =0 . 5 
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Figure 4,2 
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For any particular parameter set, however, the nominal design may 
be superior. It is also evident from Figure (4.1) - (4.2) that the 
penalty on the uncertainty should be relaxed to accommodate larger 
parameter variation. For limited parameter variation, different de- 
sign approaches nearly identical performance whereas the minimax per- 
formance sensitivity control offers better design when the parameter 
variation is large. 

4.10 CONCLUSION 

The problem of controlling a system with parameter uncertainty 
has been treated using only available output feedback. Since the 
controller is designed with incomplete state feedback, the uncer- 
tainty is likewise constrained. To achieve a design via optimiza- 
tion, a quadratic cost function involving the system state, the con- 
trol and the uncertainty vector, is defined and the optimal feed- 
back matrices relating the control and the uncertainty are chosen to 
minimize and maximize, respectively, the performance criterion. The 
resulting controller is linear, the optimal feedback matrix being 
specified by a set of simultaneous nonlinear equations. The above 
procedure usually leads to a conservative design. To meet this ob- 
jection, a sensitivity or loss criterion is defined. Minimaximi- 
zation of the sensitivity function with respect to feedback matrices 
yields a linear controller. The optimal feedback matrices must 
satisfy a set of nonlinear simultaneous algebraic equations. Some 
algorithms to solve these algebraic minimax problems and their con- 
vergence properties are discussed. An example has been treated to 
illustrate the various formulations presented in this chapter. 



It is assumed throughout this chapter that the nominal system is 
stabilized with output feedback. Even if the nominal system is sta- 
bilizable with output feedback, various minimax design procedures 
allow only certain parameters in system matrices to vary in order to 
maintain stability of the perturbed system. To relax these limita- 
tions, the required control can be generated as the response of a 
linear dynamic system whose input is the available outputs. Various 
gain matrices specifying the dynamic compensator can be determined 
in the similar as reported in previous chapter. An important limita- 
tion of the various design techniques presented in this chapter is 
the fact that the measurements are assumed to be noise-free and also 
the system is not subjected to any disturbances. In the next chap- 
ter, the stochastic version of the output feedback problem will be 
explained with and without an estimator. 
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V OPTIMAL INCOMPLETE STATE FEEDBACK CONTROLLERS 
FOR STOCHASTIC SYSTEM 

5.1 INTRODUCTION 

The problem of optimal output feedback for system with parameter 
uncertainty has been explored in the previous chapter where it was 
assumed that the measurements are noise-free and the disturbances 
in the system are negligible. This chapter will treat, among other 
things, the determination of the optimal output feedback when the 
system is excited by a white noise disturbance with and without 
measurement noise. No parameter uncertainty is assumed. Next a 
design procedure is developed for generating an optimal control as 
the output of a dynamic compensator. The input to the dynamic com- 
pensator is the available noisy output measurements of the system. 

[55] 

It is well known that if the system is linear, is excited by 
white gaussian noise and the measurement noise is also gaussian, the 
estimator and controller can be designed independently. This is due 
to so-called separation theorem. The estimator is the well-known 
Kalman filter whose dimension is equal to that of the system. The 
present chapter deals with the problem of designing a combined esti- 
mator and controller. The formulation used is more general than 
designing Kalman filter since the dimension of the estimator or 
dynamic compensator is arbitrary. Since the dimensionality of 
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Kalman filter is often a practical limitation, this problem of de- 
signing a reduced order estimator is not only challenging but may 
have substantial practical benefit. An important question to be 
answered, in this context, is whether the control is composed of 
both the estimator output and the noisy observations or only a linear 
feedback of the output of the estimator as in the design procedure 
via separation principle. The present design procedure involves 
i) a precise formulation of mathematical optimization problem, ii) 
determination of various gains specifying the dynamic compensator 
and feedback controller. The various gains should be independent 
of initial plant state so that compensator gains do not have to be 
tuned up every time the disturbance changes the plant state. Because 
of the above problem, a design procedure that is optimal only "on 
the average" will be presented. 

The state and output equations are given by 


x = Ax + Bu + Dw^ 

(5.1) 

= Cx + v^ 

(5.2) 


where the state x (an n-vector) is the signal process; the output 
y (an m-vector) is the observation process; 11 (an r-vector) is the 
control and the vectors w(jt) and v(t) are zero mean white noise pro- 
cesses of respective dimensions r and m. The covariances of these 
processes are given by 


E {w(t. )w T (t,)> 
11 

= Qi«<Vt 2 ) 

(5.3a) 

E {v(t 1 )v T (t 2 )} 

= RjMti-tj) 

(5.3b) 

E {v(t 1 )v^(t 2 )} 

= L 1 6(t 1 -t 2 ) 

(5.3c) 
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where and are positive definite matrices , and E( • ) represents 
the expected value of ( • ) . The following constraints on the control 
will be explored in this chapter: 

Case 1: Optimal output feedback with no measurement noise 

u = -Njr_ (5.4) 

0 (5.5) 

Case 2: Optimal output feedback controller with dynamic compensator 

u = Hz + Njr_ (5.6) 

where z_ is the compensator state 

£ = F£ + G£ (5.7) 

A schematic diagram is shown in Figure 5.1 

Case 3: Optimal output feedback with white measurement noise 

u = Ny_ (5.8) 

Case 4: Optimal output feedback with nonwhite measurement noise 

The controller is given by (5.8) and the output equation is 
described as ' 

£ = Cx + H£ (5.9) 

where the non-white noise £ is generated as the response of a linear 
dynamic system to the white noise v 

£ = Fz_ + Gv . (5.10) 

F, G, H, C are all specified but N is not. 

5.2 OPTIMAL OUTPUT FEEDBACK WITH NO MEASUREMENT NOISE 
5.2.1 STATEMENT AND FORMULATION OF THE PROBLEM 
1 The initial problem in this chapter is the control of the time- 
invariant system (5.1) with outputs 

y(t) = Cx(t) (5.11) 



v(t) 
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Figure 5.1 - Dynamic Compensator for Stochastic System 
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and with a controller 

u(t) = -Ny(t) = -NCx(t) 

The closed-loop system is given by 

x = [A-BNC]>t + Dw^ . 

The solution of (5.13) may be written 

x(t) = <j>(t-to)x(to) + J t <J>(t-T)Dw- L (T)dT 

to 

where the state transition matrix (t ) satisfies 


(5.12) 


(5.13) 


(5.14) 


$ = (A-BNC)4> . 

The problem is to determine N by minimizing 


r t. 


J = i E 


[x^Qx^ + u T Rujdt = j E 


(5.15) 


x T [Q+C T N T RNC]x dt 


to to 

with respect to N and subject to (5.13). (5.16) 

It is clear from (5.16) that J is determined by the initial state 
x(to) as well as matrix N. In order to make the optimum N indepen- 
dent of x(to), the expectation operation will be carried out over the 

initial conditions also. The necessary condition that N should mini- 
fy ] 

mize (5.16), requires 


li 1 1_ P 

3N ~ 2 8N L 


x T [Q+C T N T RNC]x dt 


to 


»±E 

2 


— x T CQ+C T N T RNC]x dt = 0 
3N — — 


(5.17) 


to 


The interchange of order of expectation (i.e., integration) and 
differentiation is assumed to be valid^^. 
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5.2.2 RESULT 

The optimal feedback gain 

N = R _1 B T KPC T [CPC 7 ]" 1 (5.18) 

where K and P are the solution of 

(A-BNC) T K+K(A-BNC)+Q+C T N T RNC = 0 (5.19) 

P(A-BNC) 7 + (A-BNC) T P + DQjD 7 = 0 (5.20) 

5.2.3 DERIVATION OF THE RESULT 

The partial derivative of (5.17) will be evaluated by the appli- 
cation of Lemma 4.1 and by treating the elements of N as additional 
"states" which satisfy 

N = 0 (5.21) 

Vector multiplier A^ will be used for the regular state constraint 
(5.12) and matrix multiplier will be used for constraint (5.21). 
Thus the Hamiltonian for (5.16) and (5.13) is 

H = h x 7 [Q+C T N 7 RNC]x + A 7 [(A-BNC)x + Dw , ] 
z — — -x — — 1 

= Tr [ - (Q+C 7 N T RNC)x x 7 + { (A-BNC)x + Dw} A 7 ] (5.22) 

2 _ ~ “X 

with the costate equations 


• 

A = 
-x 

3H 

dx 

= - (A-BNC) 7 ^ - (Q+C 7 N T RNC)x , 

A (t ) = 0; 
— x f 

(5.23) 

*»■ 

3H 

T T T T-T 

= - RNCx x A C A + B A x C , 

— — — X 

A(t f ) = 0. 

(5.24) 


According to the Lemma 4.1, the necessary condition and integrated 
forms of (5.24), it can be seen that 


ft. 


0 = E[l£.] = E 
9N 


[RNCx x T C T -B T A^x T C T ] dt 


(5.25) 


to 
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Thus (5.25) yields 


N = R -1 


f T T T 
B E[X x ]C dt 


— x— 


ft. 


T T -1 

CE[x x DC x dt 


to 


to 


Now assume a solution for X , of the form 

—x 


X = Kx + t) 


(5.26) 


(5.27) 


Then (5.13), (5.23) and (5.27) give 


-K = (A-BNC) T K+K(A-BNC)+Q+C T N T RNC, K(t f ) = 0 ; 


-(A-BNC) n+KDw, 


The solution of (5.29) is given by 


n(t f ) = 0 . 


n(t) = 


<|»(t-T)KDw 1 (T)dT 


(5.28) 

(5.29) 


(5.30) 


In order to complete (5.22) we how obtain the required averages. 

First, it can be easily seen from (5.14) and (5.3) that 

ft 


At T 

P = E[x x ] = iJ>(t-to)P(to)<J> (t-to) + 


(j)(t-T)DQ 1 D T 4> T (t-x)dT , 
(5.31) 


where E[x(to)x (to)] = P(to) . (5.32) 

Solving (5.31) is equivalent to solving the differential equation 


p = (a-bnc)p+p(a-bnc) t +dq 1 d t 


(5.33) 

which may be verified by differentiation of P with respect to t. 
Next , 
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E[A x T ] = KE[x x T ] + ECnx T ] 


r t 


= KP + E [{ 


$!t-x)KDw (x)dT} 


{x T (to)4» T (t-to) + 


T T T 

W 1^ T 1^ D ( i )T ^ t “ T ^ dT 1 ^ 


to 


= KF , 


(5.34) 


since the above integrals do not overlap. 
Thus (5.26) reduces to 


N = R 


-1 




T T 
B KPC dt [ 


to 


r 


to 


f CPC T dt] * 1 


(5.35) 


Ast£-*-®,t = 0»K and P are the steady state solutions of (5.28) 

r o 

and (5.33), respectively. Consequently (5.35) is indeterminate. 
Applying h* Hospital Rule as t^ ", (5.35) reduces to 

-IT T T -1 

N = R B KPC [CPC ] (5.36) 

where K and P are the steady state solutions of (5.28) and (5.33) 
respectively . 


5.2.4 COMMENTS 

It should be noted that if C = I, 

N = R-Vk . (5.37) 

(5.37) implies that the optimal feedback for the deterministic case 
(no plant disturbance ) is the same as for stochastic case (without 
measurement noise) if all the states are available for feedback. This 
is not true with incomplete feedback. (5.18) - (5.20) can be solved 
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basically with the same algorithm as suggested in Chapter 4. 

5.3 OPTIMAL DYNAMIC COMPENSATOR 

5.3.1 STATEMENT AND FORMULATION OF THE PROBLEM 
Consider now a control law 

u = Hz + N£ = Hz + NCx + Nv (5.38) 

where z_ is the state of a time- invariant dynamic compensator of fixed 
order (s^). 


£ = Fz + = Fz + GCx + Gv . (5.39) 

The problem is to determine the time-invariant matrices F(sjxs.), 
G(SjXm), H(rxs 1 ) and N(rxm) by minimizing the quadratic criterion 


ft. 


J = h 


T T 

{x Qx t u Ru }dt 


(5.40) 


to 

where Q is positive semidefinite and R is positive definite. Note 
that this formulation is general enough to include the Kalman filter. 
In order to avoid the dependence of various gain matrices to be deter- 
mined on the initial state (both the plant and the compensator), the 
expectation operation in (5.40) will also be carried out over the 
initial states by treating x(o) and zXo) to be random variable with 


E {x(o)x T (o)> = X Q 
E (z(o)£ T (o)} = Zq 


Using (5.28), (5.1) and (5.39) can be written as 


A+BNC 

GC 


BH 

F 



(5.41) 

(5.42) 


(5.43) 
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and (5.40) becomes 


J * f E 


f T T 

r T T' T T 

Q+C N 1 RNC C N i RH 



[x ,z ] 

T T 



to 

H RNC HRH J 


,z_ . 


r T T t 

+ Cv , 2^] 


r T T_ 
+ [v W^3 


r N T RNC N‘RH 


10 


,t~ti r x i 


Z J 


+ [x T z T ] 


r T T 

' C N RN 0 


HRN 


T 



N RN 0 


V 

.0 0 


w 



-r 


L % 


(5.44) 


Defining 



* .« 

A 0 

A 

B 0 

A = 


, B = 



o 

o 


0 I, 


■ 


r 


N H 

- 

I 0 

P. = 


, I = 


1 

G F. 


P 0 





A 

R 0 


Q 0 

R = 


, Q = 



P o. 


P 0. 


C = 


s = 


C 0 
0 I 
v 




, D = 


0 D 
L0 0. 


w = 


X 

z 


(5.45) 


(5.43) and (5.44) reduce to 


a a 


M = (A+BP i C)w + (EH-BPj^I )s_ 


1 

J = 2 E 


T . *T T« «■ T-t 

* Ta . rt n nn a! A*n n 


T^T-v 


(5.46) 




{w a CQ+C P^RP^Jw+w C Pj_RP 1 Is + s IP^RPjIsJdt 


to 


f V(t)dt 


(5.47) 


to 


Thus the problem is to minimize of (5.47) with respect to subject 


to (5.45). The necessary condition that'P^ should minimize J requires 



3J 

3Pi 
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rt< 


to 


dt 

3P 

1 


= 0 


(5.48) 


5.3.2 RESULTS 


The optimal gain matrix P^ is the solution of 


RP [CPC T + ~ (D+BP.I)R I + — R. (D+BP-. I) T C T ] + B T KPC T 
1 2 J. i. 2 1 


with N = 0 

where K and P are the solutions of 


0 

(5.49) 

(5.50) 


(A+BP 1 C) T K+K(A+BP 1 C)+ Q+C^RPjC = 0 


(a+bp 1 c)p+p(a+bp 1 c) t +(d+bpi)r 1 (d+bpi) t s 0 

respectively. 


(5.51) 

(5.52) 


5.3.3 DERIVATION OF THE RESULTS 

Once again the elements of the matrix P^ will be treated as addi- 


tional "states' 1 which satisfy 


P 1= 0 . 


(5.53) 


Vector multiplier A will be used for state constraint (5.46) and 


— w 


matrix multiplier A for (5.53). Thus the Hamiltonian H for (5.46) 

P 1 

and (5.47) is now 


H = 


1 T « a T T A . 1 T^T T - * 

- w [Q+C P^jCw + “ w C A P^RP^Is_ J 

1 «r X* * 1 T “ T* ■* 

+ - s IP RPaCW + - s IP a RP Is 
2- 1 1 — 2- 11- 


+ A [(A+BP n C)w + (D+BP.I)s] , 
— w -L — — 


(5.54) 
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with 


X = 


JiW A »T»A a ~ A A * T 

' |w = ■ (Q+C - c Pp^ls - (A+BP^C) a,. 


• 9H 

V = ‘ 3P, = ' 


X (t.) = 0 
— w r 


* * >• ~ T*"T * » T*. 

RP x Cw w C - RP i l£ w C _ RP^Cw s I - 


« >r“ ».rj< <T> m ■’'T T'" 

- RP.ls si - B a X w C a - B X si. 
1 — — w — w— » 


(5.55) 


A p (t f ) = 0 (5.56) 

According to the Lemma 4.1 and the necessary condition (5.48), 


E[3J/3P 1 ] = ECA p (to)] = 0 


or 


0 = RPjC 


L *P aT * A 

E[w w 1 ] dt C + RP^I 


to 


+ RP-jC 


-f T * * A 

E[w £ ]dt I + RP^I 


■t x 

to 

ft 


T a t 
E[S w ]dt C T 


to 


f E[s(t )s T (t)]dtI 


to 


~T 
+ B 


> 

--C ... 

> >'- 


t T"T T~ 

f ECX (w C +s I] dt . 
— w — — 

to 


(5.57) 


Since E[s(t)£ (t)] is infinite for white noise, (5.57) can only be 

I 

true if 

. . !rn o| 


RP 1 I = 


0 0 


= 0 


(5.58) 


(5.58) implies that unless R = 0 then 

i 

N = 0 (5.59) 

The result (5.59) is rather interesting. It implies that the obser- 
vation contaminated with white noise must be filtered irrespective of 
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the dimension of the dynamic compensator. (5.59) is, of course, true 
for the Kalman filter where its dimension is the same as that of the 
plant. 

(5.57) is the basic result which can now be simplified. For this 

purpose, a solution for X of the form 

-w 


X = Kw + n_ (5.60) 

w 

| 

is assumed. 

Substituting (5.60) into (5.55) yields 

-K = (A+BP 1 C) T K+K(A+BP 1 C)+(Q+C T P^RP 1 C), K(t f ) = 0 

(5.61) 

^ n. = -(A+BPjC^n - WD+BP^J+C^RP^Ds, n(t f )=0. (5.62) 

i 

Proceeding as previously to evaluate the averages in (5.57), define 


P = E[w w ] . 

The solutions of (5.61) and (5.62) are given by 


(5.63) 


w = <J»(t-to)w(to) + 

ft 


A A 


<fr( t-T ) ( D+BP^ ) s ( T )dT , 


to 


A A 


Txj; 


H t-T ) [K(D+BP 1 I )+C "P^RPj^I ]s ( t )dt 


(5.64) 


(5.65) 


where $(t) and t ) are the state transition matrix satisfying 

$(t) = (A+BPjCWt) • (5.66) 


*(t) = -(A+BP^Wt) . 


(5.67) 


Using (5.64) and assuming w(to) is independent of s(t ) for all time t, 
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P is given by 


P = $(t-to)P(to)$ (t-to) + 


ft 


^ T (t-T)(&fBP 1 I)R 1 (D+BP 1 I 


to 


where E[w(to)w (to)3 = P(to) 


and R^tj-tj) = EDXt^sXt^J = 


R 1 L 1 

Uj Q x J 


6(t l“V . 


(5.68) can be seen to satisfy 


P + (A+BP 1 C)P+P(A+BP i C) T +(D+BPI)R 1 (D+Bpi) T . 
Similarly 


E[s w T ] = [E{s(t)w T (to)} $ T (t-to)t 


E{s(t)s T (i 1 )}(D+BP 1 I)V(t-T i )dT 1 ) 


to 




R^i(t-T^)(l>+BP^I) <> i (t-T 1 )dT 1 


to 


A A A *W 

= -1 (D+BP^I) 1 , 


ECwCtJs^t)] 


A A A 


= (DtBPjI) |1 , 


ECn(t)w (t)D = { 


:t„t; 


t-t f )CK(D+BP 1 I)+C x P^RP 1 lDs(T)dx} 
/•t 


{w(to)<J> (t-to) + 


m A a a T T 

s (tXD+BPjI) 4> x (t- 


to 


T <t T (t-r)dT 

(5.68) 

(5.69) 

(5.70) 

(5.71) 


(5.72) 

(5.73) 

t ) dt }-0 , 

(5.74) 


I 
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since the integrals do not overlap. 


X 

E[A w T (t)3 = KE[w(t)w ( t )3 + E[n(t)w T (t)3 


= KP . 

E[A s T ] = KE[w s T ] + ECn s T ] 


(5.75) 


= j (D+BPI)^ - E | ^(t-T{k(D+BP 1 I)+C T P^RP 1 I}s(T)s T (t)dT 


j A»P <T*A A 

= - 2 c PlRPiiRi 


(5.76) 


with t - • , t = 0, K and P are steady state solution of (5.61) and 

r o 

(5.71) respectively. Under this condition and using (5.59) and (5.72) 
- (5.76), (5.57) reduces to 

A j A 

RP^ [CPC T + J (D+BP 1 I)R 1 I + j R 1 (D+BP 1 I) T C T 3 + B T KPC T = 0 (5.77) 

I 

Partitioning K and P as 


K = 


' K ll 

K 12 

P — 

P 11 P 12 

- K 12 

K 22 . 

9 * - 

p T ? 

L r 12 r 22 


(5.78) 


(5.77) reduces to equations involving the original variables: 


*«< P 12 C T ♦ i GRj) ♦ B T (K u P u . K 12 pJ)C T 


< K 12 P H + K 22 P J )cT * 0 


RHP 22 + B (K 1X P 12 + K 12 P 22 ) = 0 


K 12 P 12 + K 22 P 22 = ° 


= 0 


(5.79) 

(5.80) 

(5.81) 

(5.82) 


Using (5.59) and (5.78), (5.61) and (5.71) reduce to 
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+ K n A + K 12 GC + Q = 0 (5.83) 

H T B Tk h + F T K 12 + kJa + K 22 GC = 0 (5.84) 

hT ® Tk 12 + fTk 22 + K 12 BH + K 22 F + hTrH = 0 (5.85) 

AP-j^ + BHP^ + P 11 aT + Pi 2 hTbT + DQjD 7 = 0 (5.86) 

GCP 11 + FP 12 + P x 2 AT + P 22 hT ® T = 0 (5.87) 

GCP 12 + FP 22 + PjJc^G 1 + P 22 F T + GR-jG 7 = 0 (5.88) 


In general, evaluation of F, G, H requires simultaneous solution of 
(5.79) - (5.88). 

5.3.4 RECURSIVE ALGORITHMS FOR COMPUTING FEEDBACK. GAINS 


Various gains of the dynamic compensator can be computed using 
basically the same algorithm as reported in previous chapters. 

_.n+l j -n+l . . 

P^ and P are computed using 

Rp" +1 [CP n+1 C T + i (D+BPj +1 I)R 1 i + ^ R 1 (D+BP" +1 I) T C T ] +B T K n+1 P n+1 C 7 =0 , 

(5.89) 

(A+BP” +1 C)P n+1 + P n+1 (A+BPj +1 C) T + (D+BP n+1 i)R 1 (D+BP n+1 i), (5.90) 


where K n+ ^ is the solution of 

K^U+BP^C) + (A+BP^C)K n+1 + Q+C T P" T RP^C = 0 (5.91) 

The iteration starts with an initial guess P^° such that the augmented 
system, i.e., (A+BP^C) is stable. is the positive definite solu- 
tion of (5.91). With this value of K 1 , (5.89) and (5.90) are solved 
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5.4 


OPTIMAL OUTPUT FEEDBACK WITH WHITE MEASUREMENT NOISE 

5.4.1 PROBLEM STATEMENT 


The basic problem is to determine matrix N which minimize 
rt T T 

(5.92) 


¥ 


f T T 
(x Qx + u Ru)dt 


to 


subject to 


x = Ax + Bu + Dw 
y = Cx +v(t) 


(5.93) 

(5.94) 


and with the controller 


u = -Ny = -NCx -Nv (5.95) 

5.4.2 RESULT 

The optimal feedback gain * 

N = 0 (5.96) 

5.4.3 DERIVATION OF RESULT AND COMMENTS 

This is a special case of the previous problem and the result 
follows by setting 

F=0,G=0,H=0 (5.97) 

Note that condition (5.50) will still have to be satisfied. This 
implies (5.96) although (5.96) seems surprising but is not difficult 
to reason out. 

For any nonzero N, (5.92) is infinte since the quadratic term in the 
control involves the term E[v(t)v (t)3 which is infinte. Thus the per- 
formance index is infinite. (5.92) is finite if N = 0 for t^. is 
finite. Thus N = 0 is the optimal solution. This does not necessarily 
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imply that a nonzero optimal solution does not exist. This only points 
out the mathematical optimization problem is ill -posed. 


5.5 OPTIMAL OUTPUT FEEDBACK WITH NONWHITE MEASUREMENT NOISE 
5.5.1 STATEMENT AND FORMULATION OF THE PROBLEM 
If measurement noipe is not white, which is of course reasonable, 
thenN t 0. The controller without dynamics is given by 

u = Ny ! i (5.98) 

where the output y 

y = Cx + Hz (5.99) 

is contaminated with nonwhite noise z_ described by 

Fz + Gv . (5.100) 

F, G, H, and C are all specified. The problem is to determine N. 

Using (5.98) and (5.99), (5.1) becomes 

l 

% 

x = (A+BNC)x + BNHz + D^ 1 (5.101) 

Using (5.98) and (5.99), the performance criterion becomes 


j = K 


T T T T T T T 

{x [Q+C N RNC]x + x C x RNHz + z H RNCx 


to 


+ z T H T N T RNHz}dt. 


V 

r T T 

T T 


P — 

f T T 

Q+C N RNC 

C N RNH 


X 

(x ,z ) 

T T 

T T 




H N RNC 

H N RNH 


z 

to 





(5.102) 

Now defining 


A 

A 0 

* 

B O' 

* 

C H 

* 


X 

A = 


, B = 


, c = 


» ! = 

** 

II 



0 F 


o' 

o 


o' 

o 


U 

z 
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* 

N = 


N 0 


Q o' 

* 

’ D 0 

A 

— i 

o 


, Q = 


, D = 


, R = 


-0 o. 


o 

o. 


o' 

o 


0 oj 


V^l'V = E[s(t 1 )s T (t 2 )] = 


Ql ^ 

• L 1 *U 


<S(t l" t 2 ) » 


(5.103) 


(5.101) and (5.102) reduce to 


ft * *ft* ftft 

w = ( A+BNC )w + Ds 


(5.104) 


J = h 
2 


T ftT*T**A * 
w [C N RNC+Q]w dt. 


(5.105) 


to 

Thus the problem reduces to the minimization of (5.105) with respect 
* 

to N subject to (5.104). 


5.5.2 


RESULT 


The feedback matrix N satisfies 


AAAAAj *T***T 
RNCPC + B KPC =0 


* * 

where K and P satisfy 

A A AAA A AAA JA A AJATAAA 

K(A+BNC) + (A+BNC) K+Q+C N RNC = 0 


* * AAA j * AAA A AA ft T 
P(A+BNC) + (A+BNC )P + DR^D* = 0 

respectively. 


(5.106) 


(5.107) 

(5.108) 


5.5.3 DERIVATION AND SIMPLIFICATION OF THE RESULT 

Proceeding exactly in the same manner as in Case 1, it can be 

A 

easily seen that the feedback matrix N satisfy (5.106) - (5.108). 

A A 

Partitioning K and P as in (5.78) and expanding (5.107) and (5.108), 


it can be seen that 
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P n (A+BNC) T + P 12 H T N T B T +(A+BMC)P i1 + BNHP.J + DQ^D T = 0 (5.109) 


P.J(A+BNC) T + FP.J + GL^D T = 0 (5.110) 

i 

P 22 F T + FP 22 + GR^G 7 = 0 (5.111) 

(A+BNC) T K 1;L + K n (A+BNC)+Q+C T N T RNC = 0 (5.112) 

(BMH) T K i;l + F K 12 (A+BNC) + H T N T RNC = 0 ‘ (5.113) 

(BNH) T K 12 + F T K 22 + K^BNH + K 22 F + H T N T RNH = 0 (5.114) 

and the optimal feedback matrix N is given by 


n = -r“Vck 11 p 11 c t +k 12 p 12 h t +k 12 p 12 c t +k 12 p 22 h t ][cp 12 c t 

+CP 12 H T +HP 12 C T +HP 22 H T ] _1 . (5.115) 


If the measurement noise and plant disturbance are \mcorrelated , then 
L^ = 0. Thus (5.115) reduces to 


n = -r" 1 b t [k 11 p 11 c t +k 12 p 22 h t ]Chp 22 h t ]‘ 1 


(5.116) 


5.6 EVALUATION OF OPTIMAL COST UNDER STEADY STATE CONDITION 
Note that as t -*• 00 , t Q = 0 the integrals are in effect being 


dropped and criterion is 


j E [x T Qx. + u T Ru] = c(x,u) 


(5.117) 


In order to retain a finite cost , 


c(x^u)dt as t -*■ 00 , Q and R 

. 

to 


1 

can contain a factor (t^-t Q ) which will not affect resulting gains. 
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5.7 CONCLUSION 

A unified design procedure for optimal incomplete state feedback 
controllers for stochastic system has been presented. Various gains 
specifying the output feedback controllers with and without dynamics 
are obtained by minimizing a quadratic criterion. It has also been 
established that white noise observations must be filtered irrespec- 
tive of the dimension of the dynamic compensator. In the absence of 
any controller dynamics , th.e optimal feedback gain turns out to be 
zero if the observation process is contaminated with white noise and 
the quadratic performance index involves both the state and control. 
This merely suggests an alternative problem formulation. When the 
measurement noise is non-white, the optimal feedback matrix satisfies 
a set of nonlinear algebraic equations. In the absence of measure- 
ment noise, the optimal feedback gain must satisfy a set of algebraic 
nonlinear equations. In all the problem formulations , it is assumed 
that the feedback controller with or without dynamics stabilizes the 
system. It should be noted that the feedback matrices result from 
necessary condition of optimality. Thus the solution is not necese 
sarily globally optimal. 


O 
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VI CONCLUSION 

6.1 CONCLUSION AND LIMITATIONS OF THE PRESENT WORK 

The present thesis has attempted to present a unified design phil- 
osophy for limited state feedback control problems with parameter un- 
certainty for both deterministic and stochastic problems. 

Basically two different approaches have been suggested. In one 
approach, a linear compensator is specified, in some cases with dyna- 
mic elements. In the deterministic problem with parameter uncertainty, 
a minimax design has been accomplished by proposing an integral quad- 
ratic performance criterion which was maximized with respect to an 
uncertainty matrix and minimized by the feedback matrix. Various 
other integral quadratic criteria and design procedures have been 
examined including a sensitivity type criterion. The resulting mini- 
max controller is linear and it has been shown that minimax designs 
offer better system performance than a purely nominal design under a 
wide range of parameter variations. The various minimax procedures 
assume that the nominal system can be stabilized with output feedback. 
The stochastic problem without parameter uncertainty has been treated 
in a similar way. The stochastic problem dealt with white noie plant 
disturbance and white and colored measurement noise. Optimal limited 
state feedback controllers with and without dynamics have been formu- 
ated and optimized. The criterion is the average of an energy func- 
tion. Various optimization techniques for both stochastic and deter- 
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ministic problems result in nonlinear algebraic equations which must 
be solved recursively for the compensator matrices. 

The second approach to design is applicable for single input- single 
output systems with parameter uncertainty and uses a model of order equal 

i 

to that of the system less the number of zeros. A criterion involving 
tracking error, control, and a signal related to parameter uncertainty 
was maximized with respect to the uncertainty signal and minimized with 
respect to the control. The resulting controller is linear and uses 
only partial state feedback from states of a companion form. It has 
been shown that the plant can be stabilized with this partial state 
feedback, and the tracking error can be made arbitrarily small despite 
arbitrary parameter uncertainty, provided sufficient control energy is 
available and provided the plant is minimum phase type. The results 

t 

hold true for nonlinearities that do not involve control. In order to 
generate the control when some of the necessary states are not avail- 
able, a minimax design of reduced order dynamic compensator has been 
accomplished. The design procedure assumes noise-free measurements. 

When some of the available states are contaminated with white noise, 
an ad hoc scheme has been suggested to estimate the necessary states 
to implement the controller. 

The above two basic approaches have certain limitations. The prob- 
lem of designing a dynamic compensator for stochastic system assumes 
that the system does not involve any parameter uncertainty, although 
a minimax compensator design for stochastic system can be carried out 
in a manner similar to that reported in Chapter 3. Another limitation 
is the unavailability of efficient computational schemes for solving 
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the simultaneous algebraic equations which result from application of 
necessary conditions for an optimum controller. The minimax technique 
for single input - single output system has some drawbacks. First, it 
is difficult to extend the approach to multivariable system. The diff- 
iculty seems to be due to the fact that a suitable canonical form for 
multivariable system is not currently available. Second, the uncer- 
tainty signal is related to the system parameters in a complicated way. 
Thus an ultimate bound on parameter variation to insure system stabi- 
lity is difficult to ascertain. These problems, along with other limi- 
tations, and possible extensions of the techniques will be discussed 
in the next section. 

In spite of the various limitations of the present work, the design 
philosophy presented in this thesis makes a considerable inroad in 
handling parameter uncertainty in deterministic systems, and plant 
disturbance and measurement noise in stochastic system. It embraces a 
very challenging field ' in system theory- control of systems with para- 
meter uncertainty and disturbances using available measurements. Cer- 
tain basic investigations have been carried out in this thesis and some 
basic results have been obtained. The contribution of the present work 
will, the author hopes, stimulate further research in this field. 

6.2 OUTSTANDING PROBLEMS, POSSIBLE EXTENSIONS AND SUGGESTIONS FOR 
FURTHER RESEARCH 

Although some basic results have been obtained, the investigation is 
far from complete; however, some of the outstanding problems and sug- 
gestions for further research in this direction will be outlined below: 
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(i) The minimax design for single input - single output system as 
presented in Chapter 2 is applicable to time-invariant system. The 
present formulation can be, at best, extended to include some time- 
varying parameters. For time-varying systems, the success of the pre- 
sent technique depends much on the results of stability theory of time- 
varying systems . The present status of stability theory is not suffi- 
cient to readily establish a general result. 

(ii) Another limitation of minimax technique of Chapter 2 is the 
assumption that the system should be minimum-phase type. One way of 
approaching the problem will be to constrain the control amplitude 
leading to a saturation- type controller. The immediate question that 
arises is whether or not the tracking error can be made arbitrarily 
small with the available control amplitude. Another problem in this 
direction is to ascertain a priori the control amplitude, when very 
little is assumed to be known about the system. This, in turn, re- 
quires some more information regarding the system. 

(iii) It has been established in Chapter 3 that reduced order dy- 
namic compensator can be designed using the available measurement s . 

When some of the states (or output) are noisy, a reduced order estima- 
tor has been designed to estimate the necessary states to implement 
the control. Further research is required to establish a) what order 
dynamic compensator is necessary to stabilize the overall system, 

b) whether the reduced order compensator can give performance compara- 
ble to, that of a compensator having dimension equal to that of the 
system . 

(iv) Possibly the greatest effort should be directed to extend the 
basic concepts presented in Chapter 2 to general multivariable system. 
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This will reveal the conditions for output stabilizibility . This is 
important since the derivation of optimal output feedback controller 
with or without dynamics requires the closed-loop system to be stable. 
The difficulty in extending the present approach is due to the fact 
that no suitable canonical form for multivariable system has been found. 

(v) Obvious factors regarding minimax output feedback controller as 
presented in Chapter 4, which require further study included 

(a) Computational feasibility and convergence properties 
of various algorithms. 

(b) Existence and uniqueness of the solutions 

Cc) Stability properties of the nominal system with output 

feedback and conditions for stability if all the parameters 
in the system are allowed to vary 

Cd) Extension of the minimax design analysis to more general 
sensitivity criteria. 

Some basic questions regarding the design of dynamic compensator 
for stochastic systems include 

(1) When the dimension of the dynamic compensator is less than 
that of the controlled system, how much does the perfor- 

, mance degrade? 

(2) Is it possible to achieve separation in design of estimator 
Cor dynamic elements) and the controller, when the compenr- 
sator dimension is less than that of the system? 

C3) Can the parameter uncertainty be effectively treated for 
stochastic systems in the same manner used for determinis- 
tic problems? 
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(4) Is it possible to stabilize the system if the dimension of 
the dynamic compensator is not equal to the system order? 

(5) How should the mathematical optimization problem be refor- 
mulated to obtain a nonzero optimal feedback gain matrix 
when the observation is contaminated with white noise? A 
possible approach would be! to reformulate it as singular 
problem (integral 1 quadratic criterion penalizing the state 
only) . 

(6) Under what conditions does the algorithm presented in Chap- 
ter 5 converge? 

The answer to sane of these questions and investigation of certain of 
these factors is essential before a truly practical engineering design 
approach can be obtained. 
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